- Pass the Global command args into serviceRPC.
- Fixes error with partial cloning.
- Add partial clone test
- Include diff
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
This PR continues the work in #17125 by progressively ensuring that git
commands run within the request context.
This now means that the if there is a git repo already open in the context it will be used instead of reopening it.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Git will and can pack references into packfiles and therefore if you write/read the
files directly you will get false results. Instead you should use update-ref and
show-ref. To that end I have created three new functions in git/repo_commit.go that
will do this correctly.
Related #17191
Signed-off-by: Andrew Thornton <art27@cantab.net>
There are repeated panics in tests due to TestRepository_GetTag failing
to run properly. This happens when we attempt to reset the internal
repo for a tag which has failed to load. The problem is - the panic that
this is causing is preventing us from finding what the real error is.
This PR simply moves the failure out so we have a chance to see what
really is failing.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Prevent off-by-one error on comments on newly appended lines
There was a bug in CutDiffAroundLine whereby if a file without a terminal new line
has a patch which appends lines to it and a comment is placed on one of those lines
the comment diff will be a line out of place.
This fixes CutDiffAroundLine to simply ignore the missing terminal newline - however,
we should really improve this rendering to add a marker to say that there was a
previously missing terminal newline.
Fix #17875
Signed-off-by: Andrew Thornton <art27@cantab.net>
The current TestPatch conflict code uses a plain git apply which does not properly
account for 3-way merging. However, we can improve things using `git read-tree -m` to
do a three-way merge then follow the algorithm used in merge-one-file. We can also use
`--patience` and/or `--histogram` to generate a nicer diff for applying patches too.
Fix #13679
Fix #6417
Signed-off-by: Andrew Thornton <art27@cantab.net>
This PR contains multiple fixes. The most important of which is:
* Prevent hang in git cat-file if the repository is not a valid repository
Unfortunately it appears that if git cat-file is run in an invalid
repository it will hang until stdin is closed. This will result in
deadlocked /pulls pages and dangling git cat-file calls if a broken
repository is tried to be reviewed or pulls exists for a broken
repository.
Fix #14734
Fix #9271
Fix #16113
Otherwise there are a few small other fixes included which this PR was initially intending to fix:
* Fix panic on partial compares due to missing PullRequestWorkInProgressPrefixes
* Fix links on pulls pages due to regression from #17551 - by making most /issues routes match /pulls too - Fix #17983
* Fix links on feeds pages due to another regression from #17551 but also fix issue with syncing tags - Fix #17943
* Add missing locale entries for oauth group claims
* Prevent NPEs if ColorFormat is called on nil users, repos or teams.
The current implementation of checkBranchName is highly inefficient
involving opening the repository, the listing all of the branch names
checking them individually before then using using opened repo to get
the tags.
This PR avoids this by simply walking the references from show-ref
instead of opening the repository (in the nogogit case).
Signed-off-by: Andrew Thornton <art27@cantab.net>
This PR registers requests with the process manager and manages hierarchy within the processes.
Git repos are then associated with a context, (usually the request's context) - with sub commands using this context as their base context.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Use check attribute code to check the assigned language of a file and send that in to
chroma as a hint for the language of the file.
Signed-off-by: Andrew Thornton <art27@cantab.net>
There are multiple places where Gitea does not properly escape URLs that it is building and there are multiple places where it builds urls when there is already a simpler function available to use this.
This is an extensive PR attempting to fix these issues.
1. The first commit in this PR looks through all href, src and links in the Gitea codebase and has attempted to catch all the places where there is potentially incomplete escaping.
2. Whilst doing this we will prefer to use functions that create URLs over recreating them by hand.
3. All uses of strings should be directly escaped - even if they are not currently expected to contain escaping characters. The main benefit to doing this will be that we can consider relaxing the constraints on user names and reponames in future.
4. The next commit looks at escaping in the wiki and re-considers the urls that are used there. Using the improved escaping here wiki files containing '/'. (This implementation will currently still place all of the wiki files the root directory of the repo but this would not be difficult to change.)
5. The title generation in feeds is now properly escaped.
6. EscapePound is no longer needed - urls should be PathEscaped / QueryEscaped as necessary but then re-escaped with Escape when creating html with locales Signed-off-by: Andrew Thornton <art27@cantab.net>
Signed-off-by: Andrew Thornton <art27@cantab.net>
There is a slight race in checking of a context deadline exceed in #16467
which leads to a 500 on the repository page.
The solution is to check the error coming back from `*LogNameStatusRepoParser.Next()`
and if it is the `ContextDeadlineExceeded` break from the loop.
Fix #17314
Signed-off-by: Andrew Thornton <art27@cantab.net>
core.protectNTFS protects NTFS from files which may be difficult to remove or interact
with using the win32 api, however, it also appears to prevent such files from
being entered into the git indexes - fundamentally causing breakages with PRs that
affect these files. However, deliberately setting this to false may cause security
issues due to the remain sparse checkout of files in the merge pipeline.
The only sensible option therefore is to provide an optional setting which admins
could set which would forcibly switch this off if they are affected by this issue.
Fix #17092
Signed-off-by: Andrew Thornton <art27@cantab.net>
- Update default branch if needed
- Update protected branch if needed
- Update all not merged pull request base branch name
- Rename git branch
- Record this rename work and auto redirect for old branch on ui
Signed-off-by: a1012112796 <1012112796@qq.com>
Co-authored-by: delvh <dev.lh@web.de>
One of the biggest reasons for slow repository browsing is that we wait
until last commit information has been generated for all files in the
repository.
This PR proposes deferring this generation to a new POST endpoint that
does the look up outside of the main page request.
Signed-off-by: Andrew Thornton <art27@cantab.net>
This PR changes the compare page to make the "..." in the between branches a clickable
link. This changes the comparison type from "..." to "..". Similarly it makes the
initial compare icon clickable to switch the head and base branches.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Ignore Sync errors on pipes when doing `CheckAttributeReader.CheckPath`
* apply env patch
* Drop the Sync and fix a number of issues with the Close function
Signed-off-by: Andrew Thornton <art27@cantab.net>
* add logs for DBIndexer and CheckPath
* Fix some more closing bugs
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add test case for language_stats
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update modules/indexer/stats/db.go
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: 6543 <6543@obermui.de>
Some people still appear to report unclosed cat-files. This PR simply adds the caller
to the process descriptor for the CatFileBatch and CatFileBatchCheck calls.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Fixes #16381
Note that changes to unprotected files via the web editor still cannot be pushed directly to the protected branch. I could easily add such support for edits and deletes if needed. But for adding, uploading or renaming unprotected files, it is not trivial.
* Extract & Move GetAffectedFiles to modules/git
When the external context is cancelled it is possible for the
GitLogReader to not itself be Closed.
This PR does three things:
1. Instead of adding a plain defer it wraps the `g.Close` in a func as
`g` may change.
2. It adds the missing explicit g.Close - although the defer fix makes
this unnecessary.
3. It passes down the external context as the base context for the
GitLogReader meaning that the cancellation of the external context will
pass down automatically.
Fix #17007
Signed-off-by: Andrew Thornton <art27@cantab.net>
Replaces #16262
Replaces #16250
Replaces #14833
This PR first implements a `git check-attr` pipe reader - using `git check-attr --stdin -z --cached` - taking account of the change in the output format in git 1.8.5 and creates a helper function to read a tree into a temporary index file for that pipe reader.
It then wires this in to the language stats helper and into the git diff generation.
Files which are marked generated will be folded by default.
Fixes #14786
Fixes #12653
* make sure headGitRepo is closed on err too
* refactor
* Fix git.Blob.DataAsync(): exec cancel since we already read all bytes (close pipe since we return a NopCloser)
* Add proxy settings and support for migration and webhook
* Fix default value
* Add newline for example ini
* Add lfs proxy support
* Fix lint
* Follow @zeripath's review
* Fix git clone
* Fix test
* missgin http requests for proxy
* use empty
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
* Add info about list endpoints to CONTRIBUTING.md
* Let all list endpoints return X-Total-Count header
* Add TODOs for GetCombinedCommitStatusByRef
* Fix models/issue_stopwatch.go
* Rrefactor models.ListDeployKeys
* Introduce helper func and use them for SetLinkHeader related func
* Fix modified files list in webhooks when there is a space
There is an unfortunate bug with GetCommitFileStatus where files with
spaces are misparsed and split at the space.
There is a second bug because modern gits detect renames meaning that
this function no longer works correctly.
There is a third bug in that merge commits don't have their modified
files detected correctly.
Fix #15865
Signed-off-by: Andrew Thornton <art27@cantab.net>
Following the merging of https://github.com/go-git/go-git/pull/330 we
can now add a setting to avoid go-git reading and caching large objects.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Use `..` instead of `...` with `rev-list`. In combination with #16282 the receiver can get the correct commit. The behaviour is now like Github.
fixes #11802
There is a bug with last commit cache recursive cache where the last
commit information that refers to the current tree itself will cause a
panic due to its path ("") not being included in the expected tree entry
paths.
This PR fixes this by skipping the missing entry.
Fix #16290
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
remove log() func from gogs times and switch to proper logging
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
* Fix 500 Error with branch and tag sharing the same name #15592
Fixed 500 error while create Pull request when there are more
than one sources (branch, tag) with the same name
Fix #15592
Signed-off-by: Viktor Yakovchuk <viktor@yakovchuk.net>
* fix logging
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: 6543 <6543@obermui.de>
* More efficiently parse shas for shaPostProcessor
The shaPostProcessor currently repeatedly calls git rev-parse --verify on both backends
which is fine if there is only one thing that matches a sha - however if there are
multiple things then this becomes wildly inefficient.
This PR provides functions for both backends which are much faster to use.
Fix #16092
* Add ShaExistCache to RenderContext
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
* Improve get last commit using git log --name-status
git log --name-status -c provides information about the diff between a
commit and its parents. Using this and adjusting the algorithm to use
the first change to a path allows for a much faster generation of commit
info.
There is a subtle change in the results generated but this will cause
the results to more closely match those from elsewhere.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
* Make modules/context.Context a context.Context
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Simplify context calls
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Set the base context for requests to the HammerContext
Signed-off-by: Andrew Thornton <art27@cantab.net>
* pass context into get-last-commit
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Make commit_info cancellable
Signed-off-by: Andrew Thornton <art27@cantab.net>
* use context as context
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
* Added type sniffer.
* Switched content detection from base to typesniffer.
* Added GuessContentType to Blob.
* Moved image info logic to client.
Added support for SVG images in diff.
* Restore old blocked svg behaviour.
* Added missing image formats.
* Execute image diff only when container is visible.
* add margin to spinner
* improve BIN tag on image diffs
* Default to render view.
* Show image diff on incomplete diff.
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
Unfortunately some old repositories can have tags with empty Tagger, Commit
or Author. Go-Git variants will always have empty values for these whereas
the native git variant leaves them at nil. The simplest solution is just to
always have these set to empty Signatures.
v156 migration also makes the incorrect assumption that these cannot be empty.
Therefore add some handling to this and add logging and adjust broken
logging elsewhere in this migration.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Improve performance when there are multiple commits in the last commit cache
* read refs directly if we can
Signed-off-by: Andrew Thornton <art27@cantab.net>
Fix #15236
* Do not do 40byte conversion within ParseTreeLine
* Missed a to40ByteSHA
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
There is a weird gotcha with GetTagCommitID that because it uses git rev-list
can cause an ambiguous argument error.
This PR simply makes tags use the same code as branches.
Signed-off-by: Andrew Thornton <art27@cantab.net>
`enry.IsVendor` is kinda slow as it simply iterates across all regexps.
This PR ajdusts the regexps to combine them to make this process a
little quicker.
Related #15143
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Make auto check manual merge as a chooseable mod and add manual merge way on ui
as title, Before this pr, we use same way with GH to check manually merge.
It good, but in some special cases, misjudgments can occur. and it's hard
to fix this bug. So I add option to allow repo manager block "auto check manual merge"
function, Then it will have same style like gitlab(allow empty pr). and to compensate for
not being able to detect THE PR merge automatically, I added a manual approach.
Signed-off-by: a1012112796 <1012112796@qq.com>
* make swager
* api support
* ping ci
* fix TestPullCreate_EmptyChangesWithCommits
* Apply suggestions from code review
Co-authored-by: zeripath <art27@cantab.net>
* Apply review suggestions and add test
* Apply suggestions from code review
Co-authored-by: zeripath <art27@cantab.net>
* fix build
* test error message
* make fmt
* Fix indentation issues identified by @silverwind
Co-authored-by: silverwind <me@silverwind.io>
* Fix tests and make manually merged disabled error on API the same
Signed-off-by: Andrew Thornton <art27@cantab.net>
* a small nit
* fix wrong commit id error
* fix bug
* simple test
* fix test
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Extract out the common cat-file batch calls
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Move bleve and elastic indexers to use a common cat-file --batch when indexing
Signed-off-by: Andrew Thornton <art27@cantab.net>
* move catfilebatch to batch_reader and rename to batch_reader.go
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
Unfortunately `git log revision ... --skip=x -- path` skips the number of commits
not the number of commits relating to the path.
This PR changes the function to have a reader that reads and skips the
necessary number of commits by hand instead.
Fix #8716
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Support create single tag directly
support create tag with message from create release ui
Signed-off-by: a1012112796 <1012112796@qq.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
* CutDiffAroundLine makes the incorrect assumption that `---` and `+++` always represent part of the header of a diff.
This PR adds a flag to its parsing to prevent this problem and adds a streaming parsing technique to CutDiffAroundLine using an io.pipe instead of just sending data to an unbounded buffer.
Fix #14711
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Handle unquoted comment patch files
When making comment patches unfortunately the patch does not always quote the filename
This makes the diff --git header ambiguous again.
This PR finally adds handling for ambiguity in to parse patch
Fix #14812
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add in testing for no error
There is no way currently for CutDiffAroundLine in this test to cause an
error however, it should still be tested.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Reduce calls to git cat-file -s
There are multiple places where there are repeated calls to git cat-file
-s due to the blobs not being created with their size.
Through judicious use of git ls-tree -l and slight adjustments to the
indexer code we can avoid a lot of these calls.
* simplify by always expecting the long format
* Also always set the sized field and tell the indexer the update is sized
* Use cat-file --batch in GetLanguageStats
This PR moves to using a single cat-file --batch in GetLanguageStats
significantly reducing the number of processes spawned during language stat
processing.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* placate lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update modules/git/repo_language_stats_nogogit.go
Co-authored-by: a1012112796 <1012112796@qq.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: a1012112796 <1012112796@qq.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Somehow the test for detecting if branches are equal broke
this PR restores this functionality.
Fix #14502
Signed-off-by: Andrew Thornton <art27@cantab.net>
GetCommit() assumes that all 40 char strings are SHA1s. This leads to an
error if you try to do a PR on a branch which is 40 characters long.
This PR attempts the SHA first - and if it fails will switch to using rev-parse.
Fix #14470
Signed-off-by: Andrew Thornton <art27@cantab.net>
This PR improves HasPreviousCommit to prevent the automatic and recursive loading
of previous commits using git merge-base --is-ancestor and git rev-list
Fix #13684
Signed-off-by: Andrew Thornton <art27@cantab.net>
Memcached TTL cannot be > 30 days and if it is attempted the TTL is interpreted as
a unix timestamp.
This PR ensures that the TTL is switched to a unix timestamp in those cases.
Fix #14571
Signed-off-by: Andrew Thornton <art27@cantab.net>
Breaking the pipe is a valid way of killing a piped command and any error from
a broken cat-file batch command should be passed back up to the writer any way
therefore specifically logging it is unnecessary.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add pager to the branches page
* override pageSize if bigger than max
* Make branches commit range configurable
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: silverwind <me@silverwind.io>
* remove github.com/unknwon/com from models
* dont use "com.ToStr()"
* replace "com.ToStr" with "fmt.Sprint" where its easy to do
* more refactor
* fix test
* just "proxy" Copy func for now
* as per @lunny
* Fix git.parseTagData()
close #14092
* Add Test
* add message to test
* limit readers
* git tag -m trims and terminates with a newline
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
There is a slight bug in the commit_reader introduced in #13673 whereby
commit messages which have a final unterminated line miss their final line.
This PR fixes this.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Move last commit cache back into modules/git
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Remove go-git from the interface for last commit cache
Signed-off-by: Andrew Thornton <art27@cantab.net>
* move cacheref to last_commit_cache
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Remove go-git from routers/private/hook
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Move FindLFSFiles to pipeline
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Make no-go-git variants
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Submodule RefID
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix issue with GetCommitsInfo
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix GetLastCommitForPaths
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Improve efficiency
Signed-off-by: Andrew Thornton <art27@cantab.net>
* More efficiency
Signed-off-by: Andrew Thornton <art27@cantab.net>
* even faster
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Reduce duplication
* As per @lunny
Signed-off-by: Andrew Thornton <art27@cantab.net>
* attempt to fix drone
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix test-tags
Signed-off-by: Andrew Thornton <art27@cantab.net>
* default to use no-go-git variants and add gogit build tag
Signed-off-by: Andrew Thornton <art27@cantab.net>
* placate lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per @6543
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
After cancelling the context we still need to wait for the
command to finish otherwise zombie processes may occur
Fix #13987
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
* Multiple GitGraph improvements.
Add backend support for excluding PRs, selecting branches and files.
Fix #10327
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per @silverwind
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per @silverwind
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Only show refs in dropdown we display on the graph
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per @silverwind
Signed-off-by: Andrew Thornton <art27@cantab.net>
* use flexbox for ui header
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Move Hide Pull Request button to the dropdown
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add SHA and user pictures
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix test
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix test 2
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fixes
* async
* more tweaks
* use tabs in tmpl
Signed-off-by: Andrew Thornton <art27@cantab.net>
* remove commented thing
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix linting
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update web_src/js/features/gitgraph.js
Co-authored-by: silverwind <me@silverwind.io>
* graph tweaks
* more tweaks
* add title
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix loading indicator z-index and position
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
Go-version constraints ignore pre-releases.
Rather than change the library further this PR simply changes
the git version comparison to use simple version compare ignoring the
issue of pre-releases.
Signed-off-by: Andrew Thornton <art27@cantab.net>
This PR updates golangci-lint to the latest version 1.31.0.
The upgrade introduced a new check for which I've fixed or disabled most cases.
Signed-off-by: kolaente <k@knt.li>
* Cache last commit when pushing for big repository
* Fix bug
* detect force push
* Refactor cache push
* Finish cache last commit info when push
* Some improvements
* Fix lint
* Remove unused changes
* Move pull request test before cache
* Fix test mysql
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
If a README file is a symlink to a submodule Gitea the view branch page
will return a 500.
The underlying problem is a missed conversion of an
plumbing.ErrObjectNotFound in git/tree_blob.go.
Fix #12599
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
`GetCommitsFromIDs` is only used in one place: `LoadPushCommits` where
it expects that `c.Commits` is not nil.
This potentially nil set causes a NPE in in #12953
Fix #12953
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Don't automatically delete repository files if they are present
Prior to this PR Gitea would delete any repository files if they are
present during creation or migration. This can in certain circumstances
lead to data-loss and is slightly unpleasant.
This PR provides a mechanism for Gitea to adopt repositories on creation
and otherwise requires an explicit flag for deletion.
PushCreate is slightly different - the create will cause adoption if
that is allowed otherwise it will delete the data if that is allowed.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update swagger
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix tests and migrate overwrite
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per @lunny
Only offer to adopt or overwrite if the user can do that.
Allow the site administrator to adopt or overwrite in all
circumstances
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Use setting.Repository.DefaultBranch for the default branch
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Always set setting.Repository.DefaultBranch
Signed-off-by: Andrew Thornton <art27@cantab.net>
* update swagger
Signed-off-by: Andrew Thornton <art27@cantab.net>
* update templates
Signed-off-by: Andrew Thornton <art27@cantab.net>
* ensure repo closed
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Rewrite of adoption as per @6543 and @lunny
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Apply suggestions from code review
* update swagger
Signed-off-by: Andrew Thornton <art27@cantab.net>
* missing not
Signed-off-by: Andrew Thornton <art27@cantab.net>
* add modals and flash reporting
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Make the unadopted page searchable
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add API
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix swagger
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix swagger
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Handle empty and non-master branched repositories
Signed-off-by: Andrew Thornton <art27@cantab.net>
* placate lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
* remove commented out code
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Add configurable Trust Models
Gitea's default signature verification model differs from GitHub. GitHub
uses signatures to verify that the committer is who they say they are -
meaning that when GitHub makes a signed commit it must be the committer.
The GitHub model prevents re-publishing of commits after revocation of a
key and prevents re-signing of other people's commits to create a
completely trusted repository signed by one key or a set of trusted
keys.
The default behaviour of Gitea in contrast is to always display the
avatar and information related to a signature. This allows signatures to
be decoupled from the committer. That being said, allowing arbitary
users to present other peoples commits as theirs is not necessarily
desired therefore we have a trust model whereby signatures from
collaborators are marked trusted, signatures matching the commit line
are marked untrusted and signatures that match a user in the db but not
the committer line are marked unmatched.
The problem with this model is that this conflicts with Github therefore
we need to provide an option to allow users to choose the Github model
should they wish to.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Adjust locale strings
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per @6543
Co-authored-by: 6543 <6543@obermui.de>
* Update models/gpg_key.go
* Add migration for repository
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Right now we only compare the hostname from a submodule with the prefixURL it is viewed from to check if the submodule is hosted on the same Gitea instance. This adds an additional check to compare it against SSH_DOMAIN as well since the same Gitea instance might have a different hostname for SSH and if the submodule uses that hostname we should also detect that and link to the proper DOMAIN value.
Fixes #12747, #9756
* refactor repo_stats to use os.pipe
* woops. missing reader.
* stdout not stderr, woops
* Fix copyright date and ensure that the stderr is collected
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Set the context for CreateArchive to that of the request to ensure that archives
are only built for as long as a request is requesting them
Fix #11551
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
* go1.15
* update makefile xgo version
* fix vet issue
* update docs to version of go in use
* add TODO for asyncpreemptoff
Co-authored-by: Lauris BH <lauris@nix.lv>
Replace all calls to os.Remove/os.RemoveAll by retrying util.Remove/util.RemoveAll and remove circular dependencies from util.
Fix #12339
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: silverwind <me@silverwind.io>
#11716 reports multiple git blame processes hanging around
this was thought to be due to timeouts, however on closer look this
appears to be due to the Close() function of the BlameReader hanging
with a blocked stdout pipe.
This PR fixes this Close function to:
* Cancel the context of the cmd
* Close the StdoutReader - ensuring that the output pipe is closed
Further it makes the context of the `git blame` command a child of the
request context - ensuring that even if Close() is not called, on
cancellation of the Request the blame is command will also be cancelled.
Fixes #11716
Closes #11727
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Only write to global gitconfig if necessary
Fix #11855
Signed-off-by: Andrew Thornton <art27@cantab.net>
* placate lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
#11846 Introduced feature to show exact tag on commit view. However if a repo has no tags at all git prints out a separate and unhandled error " No names found, cannot describe anything."
Adding --always to the command makes it always use the error in the style of "fatal: no tag exactly matches" even if there are no tags at all.
Fixes #11869
Fixes #11868
* Handle more pathological branch and tag names
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix failing test
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Change language statistics to save size instead of percentage in database
Co-Authored-By: Cirno the Strongest <1447794+CirnoT@users.noreply.github.com>
* Do not exclude if only language
* Fix edge cases with special langauges
Co-authored-by: Cirno the Strongest <1447794+CirnoT@users.noreply.github.com>
Unfortunately #11614 introduced a bug whereby the initial commit of a
repository could not be seen due to there being no parent commit to
create a clear diff from.
Here we create a diffstat from the difference between the parentless SHA and the SHA of the empty tree - a constant known to git. (With thanks to @L0veSunshine for informing me of this SHA)
Thanks to @a1012112796 for initial attempt to fix.
Fix #11650
Closes #11674
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-Authored-By: L0veSunshine <xuan199651@gmail.com>
* Refactor Cron and merge dashboard tasks
* Merge Cron and Dashboard tasks
* Make every cron task report a system notice on completion
* Refactor the creation of these tasks
* Ensure that execution counts of tasks is correct
* Allow cron tasks to be started from the cron page
* golangci-lint fixes
* Enforce that only one task with the same name can be registered
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix name check
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per @guillep2k
* as per @lafriks
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add git.CommandContext variants
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Properly generate ref URLs
Tags used to not generate correct URLs (src/branch/tags/1.0.0 instead of
src/tags/1.0.0).
Also cleans up some code around it with the created helper functions.
* Fix formatting and create migration
* Add copyright head to utils_test
* Use a raw query for the ref migration
* Remove semicolon
* Quote column and table names in migration SQL
* Change || to CONCAT, since MSSQL does not support ||
* Make migration engine aware
* Add missing import
* Move ref EndName and URL to the issue service
* Fix tests
* Add test for commit refs
* Update issue.go
* Use the right command for building JavaScript bundles
* Prepare for merge
* Check for refs/* before prepending in migration
* Update services/issue/issue_test.go
* Update modules/git/utils_test.go
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: techknowlogick <matti@mdranta.net>
* Allow compare page to look up base, head, own-fork, forkbase-of-head
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per @guillep2k
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update routers/repo/compare.go
* as per @guillep2k
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Rationalise the names a little
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Rationalise the names a little (2)
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix 500 with fork of fork
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Prevent 500 on compare different trees
Signed-off-by: Andrew Thornton <art27@cantab.net>
* dotdotdot is perfectly valid in both usernames and repo names
Signed-off-by: Andrew Thornton <art27@cantab.net>
* ensure we can set the head and base repos too
Signed-off-by: Andrew Thornton <art27@cantab.net>
* ensure we can set the head and base repos too (2)
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
* only set headRepo == baseRepo if isSameRepo
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Manage port in submodule refurl
Fix #11304
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
* URLJoin is causes a cyclic dependency and possibly isn't what what we want anyway
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Protect against leading .. in scp syntax
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
* Support unicode emojis and remove emojify.js
This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea.
This works in a few ways:
First it adds emoji parsing support into gitea itself. This allows us to
* Render emojis from valid alias (😄)
* Detect unicode emojis and let us put them in their own class with proper aria-labels and styling
* Easily allow for custom "emoji"
* Support all emoji rendering and features without javascript
* Uses plain unicode and lets the system render in appropriate emoji font
* Doesn't leave us relying on external sources for updates/fixes/features
That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also)
For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method.
The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released.
I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens.
I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others.
Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary.
Fixes: https://github.com/go-gitea/gitea/issues/9182
Fixes: https://github.com/go-gitea/gitea/issues/8974
Fixes: https://github.com/go-gitea/gitea/issues/8953
Fixes: https://github.com/go-gitea/gitea/issues/6628
Fixes: https://github.com/go-gitea/gitea/issues/5130
* add new shared function emojiHTML
* don't increase emoji size in issue title
* Update templates/repo/issue/view_content/add_reaction.tmpl
Co-Authored-By: 6543 <6543@obermui.de>
* Support for emoji rendering in various templates
* Render code and review comments as they should be
* Better way to handle mail subjects
* insert unicode from tribute selection
* Add template helper for plain text when needed
* Use existing replace function I forgot about
* Don't include emoji greater than Unicode Version 12
Only include emoji and aliases in JSON
* Update build/generate-emoji.go
* Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have
* final updates
* code review
* code review
* hard code gitea custom emoji to match previous behavior
* Update .eslintrc
Co-Authored-By: silverwind <me@silverwind.io>
* disable preempt
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
As seen in trouble shooting #11032 the new feature of Go 1.14 is causing several second delays in startup in certain situations. Debugging shows it spending several seconds handling SIGURG commands during init:
```
6922:04:51.984234 trace init() ./modules/queue/unique_queue_wrapped.go
remote: ) = 69 <0.000012>
remote: [pid 15984] 22:04:51 write(1, "\ttime taken: 236.761\302\265s\n\n", 25 time taken: 236.761µs
remote:
remote: ) = 25 <0.000011>
remote: [pid 15984] 22:04:51 --- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=15984, si_uid=0} ---
remote: [pid 15984] 22:04:52 --- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=15984, si_uid=0} ---
remote: [pid 15984] 22:04:52 --- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=15984, si_uid=0} ---
```
This causes up to 20 seconds added to a push in some cases as it happens for each call of the gitea hook command. This is likely the cause of #10661 as well and would start to effect users once we release 1.12 which would be the first release compiled with Go 1.14. I suspect this is just a slight issue with the upstream implementatation as there have been a few very similar bugs fixed and reported:
https://github.com/golang/go/issues/37741https://github.com/golang/go/issues/37942
We should revisit this in the future and see if a newer version of Go has solved it, but for now disable this option in the environment that gitea hook runs in to avoid it.
