This PR follows #21535 (and replace #22592)
## Review without space diff
https://github.com/go-gitea/gitea/pull/22678/files?diff=split&w=1
## Purpose of this PR
1. Make git module command completely safe (risky user inputs won't be
passed as argument option anymore)
2. Avoid low-level mistakes like
https://github.com/go-gitea/gitea/pull/22098#discussion_r1045234918
3. Remove deprecated and dirty `CmdArgCheck` function, hide the `CmdArg`
type
4. Simplify code when using git command
## The main idea of this PR
* Move the `git.CmdArg` to the `internal` package, then no other package
except `git` could use it. Then developers could never do
`AddArguments(git.CmdArg(userInput))` any more.
* Introduce `git.ToTrustedCmdArgs`, it's for user-provided and already
trusted arguments. It's only used in a few cases, for example: use git
arguments from config file, help unit test with some arguments.
* Introduce `AddOptionValues` and `AddOptionFormat`, they make code more
clear and simple:
* Before: `AddArguments("-m").AddDynamicArguments(message)`
* After: `AddOptionValues("-m", message)`
* -
* Before: `AddArguments(git.CmdArg(fmt.Sprintf("--author='%s <%s>'",
sig.Name, sig.Email)))`
* After: `AddOptionFormat("--author='%s <%s>'", sig.Name, sig.Email)`
## FAQ
### Why these changes were not done in #21535 ?
#21535 is mainly a search&replace, it did its best to not change too
much logic.
Making the framework better needs a lot of changes, so this separate PR
is needed as the second step.
### The naming of `AddOptionXxx`
According to git's manual, the `--xxx` part is called `option`.
### How can it guarantee that `internal.CmdArg` won't be not misused?
Go's specification guarantees that. Trying to access other package's
internal package causes compilation error.
And, `golangci-lint` also denies the git/internal package. Only the
`git/command.go` can use it carefully.
### There is still a `ToTrustedCmdArgs`, will it still allow developers
to make mistakes and pass untrusted arguments?
Generally speaking, no. Because when using `ToTrustedCmdArgs`, the code
will be very complex (see the changes for examples). Then developers and
reviewers can know that something might be unreasonable.
### Why there was a `CmdArgCheck` and why it's removed?
At the moment of #21535, to reduce unnecessary changes, `CmdArgCheck`
was introduced as a hacky patch. Now, almost all code could be written
as `cmd := NewCommand(); cmd.AddXxx(...)`, then there is no need for
`CmdArgCheck` anymore.
### Why many codes for `signArg == ""` is deleted?
Because in the old code, `signArg` could never be empty string, it's
either `-S[key-id]` or `--no-gpg-sign`. So the `signArg == ""` is just
dead code.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Change all license headers to comply with REUSE specification.
Fix #16132
Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
Use Unicode placeholders to replace HTML tags and HTML entities first, then do diff, then recover the HTML tags and HTML entities. Now the code diff with highlight has stable behavior, and won't emit broken tags.
* Move access and repo permission to models/perm/access
* fix test
* fix git test
* Move functions sequence
* Some improvements per @KN4CK3R and @delvh
* Move issues related code to models/issues
* Move some issues related sub package
* Merge
* Fix test
* Fix test
* Fix test
* Fix test
* Rename some files
This PR continues the work in #17125 by progressively ensuring that git
commands run within the request context.
This now means that the if there is a git repo already open in the context it will be used instead of reopening it.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Fix #17514
Given the comments I've adjusted this somewhat. The numbers of characters detected are increased and include things like the use of U+300 to make à instead of à and non-breaking spaces.
There is a button which can be used to escape the content to show it.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Gwyneth Morgan <gwymor@tilde.club>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Allow Loading of Diffs that are too large
This PR allows the loading of diffs that are suppressed because the file
is too large. It does not handle diffs of files which have lines which
are too long.
Fix #17738
Signed-off-by: Andrew Thornton <art27@cantab.net>
Unfortunately due to a misread on my behalf I missed that git diff only learned
--skip-to in version 2.31.0. Thus this functionality was not working on older versions
of git.
This PR adds a handler that simply allows for us to skip reading the diffs until
we find the correct file to skip to.
Fix #17731
Signed-off-by: Andrew Thornton <art27@cantab.net>
Use check attribute code to check the assigned language of a file and send that in to
chroma as a hint for the language of the file.
Signed-off-by: Andrew Thornton <art27@cantab.net>
This PR changes the compare page to make the "..." in the between branches a clickable
link. This changes the comparison type from "..." to "..". Similarly it makes the
initial compare icon clickable to switch the head and base branches.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
remove log() func from gogs times and switch to proper logging
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
* CutDiffAroundLine makes the incorrect assumption that `---` and `+++` always represent part of the header of a diff.
This PR adds a flag to its parsing to prevent this problem and adds a streaming parsing technique to CutDiffAroundLine using an io.pipe instead of just sending data to an unbounded buffer.
Fix #14711
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Handle unquoted comment patch files
When making comment patches unfortunately the patch does not always quote the filename
This makes the diff --git header ambiguous again.
This PR finally adds handling for ambiguity in to parse patch
Fix #14812
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add in testing for no error
There is no way currently for CutDiffAroundLine in this test to cause an
error however, it should still be tested.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Gitea runs diff on highlighted code fragment for each line in order to provide
code highlight diffs. Unfortunately this diff algorithm is not aware that span tags
and entities are atomic and cannot be split.
The current fixup code makes some attempt to fix these broken tags however, it cannot
handle situations where a tag is split over multiple blocks.
This PR provides a more algorithmic fixup mechanism whereby spans and entities are
completely coalesced into their respective blocks.
This may result in a incompletely reduced diff but - it will definitely prevent the
broken entities and spans that are currently possible.
As a result of this fixup several inconsistencies were discovered in our testcases
and these were also fixed.
Fix #14231
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Handle incomplete diff files properly
The code for parsing diff hunks has a bug whereby a very long line
in a very long diff would not be completely read leading to an unexpected
character.
This PR ensures that the line is completely cleared
Fix #13602
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Also allow git max line length <4096
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add test case
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Alternative fix for HTML diff entity split
This commit both reverts PR #13357 and uses the exiting implementation alredy used for spans to fix the same issue. That PR duplicates most of logic that is already present elsewhere and still was failing for some cases. This should be simpler as it uses the existing logic that already works for <span>s being split apart.
Added both test cases as well.
* Update gitdiff_test.go
* fmt
* entity can have uppercase letter, also add detailed comment per @zeripath
* When creating line diffs do not split within an html entity
Fix #13342
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add test case
Signed-off-by: Andrew Thornton <art27@cantab.net>
* improve test
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Add better error checking for inline html diff code
A better fix for #13191 which cleans up this code a bit and adds basic checking which should avoid writing broken HTML in future situations.
* Update gitdiff_test.go
* better regex
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Fix error in diff html rendering
Was missing an optional whitespace check in regex. Also noticed a rare case where diff.Type == Equal would be empty and thus get a newline attached. Fixed that too.
Fixes #13177
* Update services/gitdiff/gitdiff.go
Co-authored-by: zeripath <art27@cantab.net>
* Update gitdiff_test.go
* fmt
Co-authored-by: zeripath <art27@cantab.net>
* Fix diff skipping lines
ParsePatch previously just skipped all lines that start with "+++ " or "--- "
and makes no attempt to see these lines in context.
This PR rewrites ParsePatch to pay attention to context and position
within a patch, ensuring that --- and +++ are only skipped if
appropriate.
This PR also fixes several issues with incomplete files.
Fix https://codeberg.org/Codeberg/Community/issues/308
Fix #13153
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add testcase
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix comment
* simplify error handling
Signed-off-by: Andrew Thornton <art27@cantab.net>
* never return io.EOF
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
* Finally fix diff names
#12771 attempted to fix diff by avoiding the git diff line as
it is possible to have an ambiguous line here.
#12254 attempted to fix diff by assuming that names would quoted
if they needed to be and if one was quoted then both would be.
Both of these were wrong.
I have now discovered `--src-prefix` and `--dst-prefix` which
means that we can set this in such a way to force the git diff
to always be unambiguous.
Therefore this PR rollsback most of the changes in #12771 and
uses these options to fix this.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update services/gitdiff/gitdiff.go
* Update services/gitdiff/gitdiff.go
* Update modules/repofiles/temp_repo.go
* fix test
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Following further testing it has become apparent that the diff line
cannot be used to determine filenames for diffs with any sort of predictability
the answer therefore is to use the other lines that are provided with a diff
Fix #12768
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix diff path unquoting
services/gitdiff/gitdiff.go whereby there it assumed that the path would
always be quoted on both sides
This PR simplifies the code here and uses fmt.Fscanf to parse the
strings as necessary.
Fix #12546
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add testcase as per @mrsdizzie
Signed-off-by: Andrew Thornton <art27@cantab.net>
Make previous fix from #12238 more robust since I saw a case where a diff changes only a single character in a chroma class instead of the entire thing. Add another more complicated test to match.
Co-authored-by: Lauris BH <lauris@nix.lv>
* Fix Syntax highlight for token change in added/deleted code
For diffs we first syntax highlight the before and after line then use a 3rd party diff library to find the difference in them and create a substring based on that, which we then highlight a 2nd time to show the specific difference within a line that has changed. In a specific case if the diffrence also changes the chroma class it will split in the middle of the attr and cause broken HTML:
```
<span class="nx">oldtext<span>
<span class="k">var newtext<span>
```
Will then split on
```
<span class="
```
Where the difference starts, and produce something broken like:
```
<span class="<span class="removed-code">nx"oldtext</span></span
```
Fix that by detecting when it happens and putting the HTML back together properly before highlighting the added/deleted code.
Fixes #12235
* fix lint
* apply fix to all diff sections. Also handle case where insert/remove starts with a closing span
* Add a test for this new code
* remove comment
Co-authored-by: Lauris BH <lauris@nix.lv>
* Server-side syntax hilighting for all code
This PR does a few things:
* Remove all traces of highlight.js
* Use chroma library to provide fast syntax hilighting directly on the server
* Provide syntax hilighting for diffs
* Re-style both unified and split diffs views
* Add custom syntax hilighting styling for both regular and arc-green
Fixes #7729
Fixes #10157
Fixes #11825
Fixes #7728
Fixes #3872
Fixes #3682
And perhaps gets closer to #9553
* fix line marker
* fix repo search
* Fix single line select
* properly load settings
* npm uninstall highlight.js
* review suggestion
* code review
* forgot to call function
* fix test
* Apply suggestions from code review
suggestions from @silverwind thanks
Co-authored-by: silverwind <me@silverwind.io>
* code review
* copy/paste error
* Use const for highlight size limit
* Update web_src/less/_repository.less
Co-authored-by: Lauris BH <lauris@nix.lv>
* update size limit to 1MB and other styling tweaks
* fix highlighting for certain diff sections
* fix test
* add worker back as suggested
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
* Some refactor on git diff and ignore getting commit information failed on migrating pull request review comments
* fix test
* fix lint
* Change error log to warn
* fix typo
* Migrate reviews when migrating repository from github
* fix lint
* Added test and migration when external user login
* fix test
* fix commented state
* Some improvements
* fix bug when get pull request and ref original author on code comments
* Fix migrated line; Added comment for review
* Don't load all pull requests attributes
* Fix typo
* wrong change copy head
* fix tests
* fix reactions
* Fix test
* fix fmt
* fix review comment reactions
