#1854  Unintended code diff changes?
Closed
jbauer opened 2 weeks ago

After upgrading to 10.4.0 I noticed two strange code diff behaviors in our pull requests.

  1. If a PR contains a file that has changed by removing indentation white spaces in otherwise empty lines only, then OneDev will show no diff but instead the message "Content is identical". I think the changes in white space should be shown because that is what all other git tools do and thus it feels a bit weird. The only situation I am aware of that you can commit a file with identical content is if you modify file permissions.
  2. If a file has german umlauts (ä, ü, ö) and that file is part of a commit in a PR then OneDev shows the line containing the german umlaut as modified even though other git tools do not and the committer hasn't changed that line. Looks like an encoding issue. See image encoding.png

Maybe some libraries or Java version has been updated in OneDev server docker container that cause this?

Robin Shen commented 2 weeks ago

For issue 1, a bug is filed: #1856

For issue 2, I can not get it reproduced. Please create a project demonstrating this.

jbauer commented 1 week ago

With regard to 2:

I did the following so far:

  • Checked out a commit before the PR was merged and opened the file in ImHex app. It shows ü has bytes c3 bc which is LATIN SMALL LETTER U WITH DIAERESIS
  • Checked out a commit after the PR had been merged and operned the file again. It shows ü has still bytes c3 bc. So the file bytes haven't really changed in that location.
  • Restored the PR branch in OneDev, checked it out on my host, created a new local branch, pushed it to OneDev and made a new testing PR in OneDev. The issue occurred again in OneDev PR UI. This time the branch has been pushed from a different host and git tool.
  • With the testing PR opened I entered the OneDev docker container on the server, navigated to the project and then went to git/refs/pulls/<id>. I then used git-diff to compare commits stored in base and head files and the diff DOES NOT show a change in that line. Kind of logical because bytes haven't changed on disk. Still, I see changes in OneDev in that line.

So it seems to be a pure OneDev problem. However I cannot reproduce it manually here in your installation using the OneDev UI. But I can reproduce it using the PR branch in my installation.

The container runs with locale en_US.UTF-8 and java -XshowSettings:properties -version also outputs UTF-8.

So no idea how to figure it out, except copying everything and run OneDev from the IDE with breakpoints.

jbauer commented 1 week ago

@robin I noticed in my installation the file in question is treated by OneDev as ISO 8859-1 encoded file, even though ü is clearly encoded as UTF-8 bytes (opened it in hex editor). I then used OneDev UI and viewed some files in Code -> Files and it seems OneDev treats some files as ISO 8859-1 and some other files as UTF-8 even though all developers use UTF-8 encoding in their tools. We try to avoid using german umlauts in code so if you guess the encoding based on the content it could very well be that most files can be treated as ISO 8859-1 or UTF-8 because their encoding would be the same. However in string literals or comments we use german umlauts from time to time.

So for some reason in the PR in question OneDev uses the file of the PR base commit in UTF-8 encoding (because in the image above the ü is displayed in red) and the pushed changes (PR head) for that file are treated as ISO 8859-1 by OneDev only (git tools do not see any changes, only OneDev). Thus the diff now displays the change for ü in OneDev only.

How does OneDev detect the file encoding? Is that a setting somewhere? As said before all developer tools and JDK of OneDev container itself use UTF-8 encoding.

Robin Shen commented 1 week ago

Please reproduce this with a test OneDev set up, and attach the database and site/projects folder.

jbauer commented 1 week ago

@robin After looking at your server code the reason is actually relatively simple. See description and code diff (the very last line in the diff): https://code.onedev.io/jbauer/~pulls/3

Robin Shen commented 6 days ago

Thanks for the info. Will be fixed in next patch release.

Robin Shen changed state to 'Closed' 6 days ago
Previous Value Current Value
Open
Closed
issue 1 of 1
Type
Question
Priority
Normal
Assignee
Labels
No labels
Issue Votes (0)
Watchers (3)
Reference
onedev/server#1854
Please wait...
Page is in error, reload to recover