* Add a storage layer for attachments
* Fix some bug
* fix test
* Fix copyright head and lint
* Fix bug
* Add setting for minio and flags for migrate-storage
* Add documents
* fix lint
* Add test for minio store type on attachments
* fix test
* fix test
* Apply suggestions from code review
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
* Add warning when storage migrated successfully
* Fix drone
* fix test
* rebase
* Fix test
* display the error on console
* Move minio test to amd64 since minio docker don't support arm64
* refactor the codes
* add trace
* Fix test
* remove log on xorm
* Fi download bug
* Add a storage layer for attachments
* Add setting for minio and flags for migrate-storage
* fix lint
* Add test for minio store type on attachments
* Apply suggestions from code review
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
* Fix drone
* fix test
* Fix test
* display the error on console
* Move minio test to amd64 since minio docker don't support arm64
* refactor the codes
* add trace
* Fix test
* Add URL function to serve attachments directly from S3/Minio
* Add ability to enable/disable redirection in attachment configuration
* Fix typo
* Add a storage layer for attachments
* Add setting for minio and flags for migrate-storage
* fix lint
* Add test for minio store type on attachments
* Apply suggestions from code review
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
* Fix drone
* fix test
* Fix test
* display the error on console
* Move minio test to amd64 since minio docker don't support arm64
* don't change unrelated files
* Fix lint
* Fix build
* update go.mod and go.sum
* Use github.com/minio/minio-go/v6
* Remove unused function
* Upgrade minio to v7 and some other improvements
* fix lint
* Fix go mod
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-authored-by: Tyler <tystuyfzand@gmail.com>
* Clarify documentation of SKIP_VERIFY
The documentation clearly documents the empty value as the default,
however at least one user reported this as being unclear. Mark values
explicitly so it is clear what values it can take. This clarifies that
an empty value in fact leaves certificate verification enabled, whereas
it has to be explicitly set to true to disable certificate verification.
Resolves: #12117
Signed-off-by: Alexander Scheel <alexander.m.scheel@gmail.com>
* Update docs/content/doc/advanced/config-cheat-sheet.en-us.md
Co-authored-by: mrsdizzie <info@mrsdizzie.com>
* Update custom/conf/app.example.ini
Co-authored-by: mrsdizzie <info@mrsdizzie.com>
* Update custom/conf/app.example.ini
Co-authored-by: mrsdizzie <info@mrsdizzie.com>
Co-authored-by: mrsdizzie <info@mrsdizzie.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Move EventSource to use a SharedWorker. This prevents issues with HTTP/1.1
open browser connections from preventing gitea from opening multiple tabs.
Also allow setting EVENT_SOURCE_UPDATE_TIME to disable EventSource updating
Fix #11978
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Rename custom/conf/app.ini.sample to custom/conf/app.sample.ini for better syntax light on editor
* rename to app.example.ini
* per @6543 's comment, update all references on docs
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
This removes the jQuery plugin as well as the associated config options.
Native input[type=date] does not require a language attribute as it is
localized by default, except for the placeholder attribute for which I
currently piggy-back the repo.issues.due_date_form localization option.
Implementation should pretty much match GH. Of note is that Safari does
not provide a UI for this input type, but I don't think providing one is
neccessary and GH did not bother either.
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Allow site admin to disable mirrors
Signed-off-by: jolheiser <john.olheiser@gmail.com>
* No need to run through Safe
Signed-off-by: jolheiser <john.olheiser@gmail.com>
* Clarify only disabling NEW mirrors
Signed-off-by: jolheiser <john.olheiser@gmail.com>
* Apply suggestions from @guillep2k
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
* Fix chardet test and add ordering option
Signed-off-by: Andrew Thornton <art27@cantab.net>
* minor fixes
Signed-off-by: Andrew Thornton <art27@cantab.net>
* remove log
Signed-off-by: Andrew Thornton <art27@cantab.net>
* remove log2
Signed-off-by: Andrew Thornton <art27@cantab.net>
* only iterate through top results
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update docs/content/doc/advanced/config-cheat-sheet.en-us.md
* slight restructure of for loop
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Add warning to mailer documentation about authentication
References #7966
Signed-off-by: Andrew Thornton <art27@cantab.net>
* As per @guillep2k and @mrsdizzie
* as per @mrsdizzie
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
GH has different HardBreaks behaviour for markdown comments and documents.
Comments have hard breaks and documents have soft breaks - therefore Gitea's rendering will always be different from GH's if we only provide one setting.
Here we split the setting in to two - one for documents and one for comments and other things.
Signed-off-by: Andrew Thornton art27@cantab.net
Changes to index.js as per @silverwind
Co-authored-by: silverwind <me@silverwind.io>
Changes to docs as per @guillep2k
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
* Make sure that sendmail processes register with the process manager
* Provide a timeout for these (initially of 5 minutes)
* Add configurable value and tie in to documentation
* Tie in to the admin config page.
Signed-off-by: Andrew Thornton <art27@cantab.net>
In #9888, it was reported that my earlier pull request #9075 didn't quite function as expected. I was quite hopeful the `ValuesWithShadow()` worked as expected (and, I thought my testing showed it did) but I guess not. @zeripath proposed an alternative syntax which I like:
```ini
[markup.sanitizer.1]
ELEMENT=a
ALLOW_ATTR=target
REGEXP=something
[markup.sanitizer.2]
ELEMENT=a
ALLOW_ATTR=target
REGEXP=something
```
This was quite easy to adopt into the existing code. I've done so in a semi-backwards-compatible manner:
- The value from `.Value()` is used for each element.
- We parse `[markup.sanitizer]` and all `[markup.sanitizer.*]` sections and add them as rules.
This means that existing configs will load one rule (not all rules). It also means people can use string identifiers (`[markup.sanitiser.KaTeX]`) if they prefer, instead of numbered ones.
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
* Support unicode emojis and remove emojify.js
This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea.
This works in a few ways:
First it adds emoji parsing support into gitea itself. This allows us to
* Render emojis from valid alias (😄)
* Detect unicode emojis and let us put them in their own class with proper aria-labels and styling
* Easily allow for custom "emoji"
* Support all emoji rendering and features without javascript
* Uses plain unicode and lets the system render in appropriate emoji font
* Doesn't leave us relying on external sources for updates/fixes/features
That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also)
For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method.
The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released.
I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens.
I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others.
Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary.
Fixes: https://github.com/go-gitea/gitea/issues/9182
Fixes: https://github.com/go-gitea/gitea/issues/8974
Fixes: https://github.com/go-gitea/gitea/issues/8953
Fixes: https://github.com/go-gitea/gitea/issues/6628
Fixes: https://github.com/go-gitea/gitea/issues/5130
* add new shared function emojiHTML
* don't increase emoji size in issue title
* Update templates/repo/issue/view_content/add_reaction.tmpl
Co-Authored-By: 6543 <6543@obermui.de>
* Support for emoji rendering in various templates
* Render code and review comments as they should be
* Better way to handle mail subjects
* insert unicode from tribute selection
* Add template helper for plain text when needed
* Use existing replace function I forgot about
* Don't include emoji greater than Unicode Version 12
Only include emoji and aliases in JSON
* Update build/generate-emoji.go
* Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have
* final updates
* code review
* code review
* hard code gitea custom emoji to match previous behavior
* Update .eslintrc
Co-Authored-By: silverwind <me@silverwind.io>
* disable preempt
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
* backwards compatibility typo
* Unlist description and separate from list by an empty line
* redis needs password edit
* Update docs/content/doc/advanced/config-cheat-sheet.en-us.md
Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
* Use AJAX for notifications table
Signed-off-by: Andrew Thornton <art27@cantab.net>
* move to separate js
Signed-off-by: Andrew Thornton <art27@cantab.net>
* placate golangci-lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add autoupdating notification count
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix wipeall
Signed-off-by: Andrew Thornton <art27@cantab.net>
* placate tests
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Try hidden
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Try hide and hidden
Signed-off-by: Andrew Thornton <art27@cantab.net>
* More auto-update improvements
Only run checker on pages that have a count
Change starting checker to 10s with a back-off to 60s if there is no change
Signed-off-by: Andrew Thornton <art27@cantab.net>
* string comparison!
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per @silverwind
Signed-off-by: Andrew Thornton <art27@cantab.net>
* add configurability as per @6543
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add documentation as per @6543
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Use CSRF header not query
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Further JS improvements
Fix @etzelia update notification table request
Fix @silverwind comments
Co-Authored-By: silverwind <me@silverwind.io>
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Simplify the notification count fns
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: silverwind <me@silverwind.io>
Some OAuth2 providers return quite large structured tokens >32767 bytes.
Gitea currently has a fixed maximum of 32767 bytes for these and
unfortunately due to the convoluted nature of the dependent libraries the
error returned is rather opaque.
Here we manage the error a little better - detecting the rather opaque
github.com/gorilla/securecookie.errEncodedValueTooLong and converting
it to a more readable error.
Further we provide a configurable option to increase the maximum size of
the provided OAuth2 tokens.
Fix #9907
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Across several files, references to cron.update_migration_post_id are made, although the actual setting is called cron.update_migration_poster_id according to modules/setting/cron.go (10e2f29144/modules/setting/cron.go (L54)).
Co-authored-by: Antoine GIRARD <sapk@users.noreply.github.com>
* Add more logging in the LFS server
Adds more logging in the LFS server and stops sending internal server
error information to the client
* Add LFS Lock cursor implementation
* Simplify Claims in LFS and remove the float64 casts
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Move langauge detection to separate module to be more reusable
Add option to disable vendored file exclusion from file search
Allways show all language stats for search
* Upgrade levelqueue to version 0.2.0
This adds functionality for Unique Queues
* Add UniqueQueue interface and functions to create them
* Add UniqueQueue implementations
* Move TestPullRequests over to use UniqueQueue
* Reduce code duplication
* Add bytefifos
* Ensure invalid types are logged
* Fix close race in PersistableChannelQueue Shutdown
* Cache last commit to accelerate the repository directory page visit
* Default use default cache configuration
* add tests for last commit cache
* Simplify last commit cache
* Revert Enabled back
* Change the last commit cache default ttl to 8760h
* Fix test
* Add support for database schema
* Require setting search_path for the db user
* Add schema setting to admin/config.tmpl
* Use a schema different from default for psql tests
* Update postgres scripts to use custom schema
* Update to xorm/core 0.7.3 and xorm/xorm c37aff9b3a
* Fix migration test
Co-authored-by: Antoine GIRARD <sapk@users.noreply.github.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Make CertFile and KeyFile relative to CustomPath
The current code will absolute CertFile and KeyFile against the current working directory. This is quite unexpected for users. This code makes relative paths absolute against the CustomPath.
Fix #4196
* Improve error reporting when reading certificates
* Apply suggestions from code review
Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
* Delay printing hook statuses until after 1 second
* Move to a 5s delay, wrapped writer structure and add config
* Update cmd/hook.go
* Apply suggestions from code review
* Update cmd/hook.go
Co-authored-by: Antoine GIRARD <sapk@users.noreply.github.com>
* Refactor
Signed-off-by: jolheiser <john.olheiser@gmail.com>
* Add push-create to SSH serv
Signed-off-by: jolheiser <john.olheiser@gmail.com>
* Cannot push for another user unless admin
Signed-off-by: jolheiser <john.olheiser@gmail.com>
* Get owner in case admin pushes for another user
Signed-off-by: jolheiser <john.olheiser@gmail.com>
* Set new repo ID in result
Signed-off-by: jolheiser <john.olheiser@gmail.com>
* Update to service and use new org perms
Signed-off-by: jolheiser <john.olheiser@gmail.com>
* Move pushCreateRepo to services
Signed-off-by: jolheiser <john.olheiser@gmail.com>
* Fix import order
Signed-off-by: jolheiser <john.olheiser@gmail.com>
* Changes for @guillep2k
* Check owner (not user) in SSH
* Add basic tests for created repos (private, not empty)
Signed-off-by: jolheiser <john.olheiser@gmail.com>
* Default NO_REPLY_ADDRESS to DOMAIN
The default NO_REPLY_ADDRESS was previously noreply.example.org
This PR changes this to default to the default DOMAIN for gitea.
* Change default to noreply.%(DOMAIN)s
* Change default to "noreply" + Domain
* Support custom sanitization policy
Allowing the gitea administrator to configure sanitization policy allows
them to couple external renders and custom templates to support more
markup. In particular, the `pandoc` renderer allows generating KaTeX
annotations, wrapping them in `<span>` elements with class `math` and
either `inline` or `display` (depending on whether or not inline or
block mode was requested).
This iteration gives the administrator whitelisting powers; carefully
crafted regexes will thus let through only the desired attributes
necessary to support their custom markup.
Resolves: #9054
Signed-off-by: Alexander Scheel <alexander.m.scheel@gmail.com>
* Document new sanitization configuration
- Adds basic documentation to app.ini.sample,
- Adds an example to the Configuration Cheat Sheet, and
- Adds extended information to External Renderers section.
Signed-off-by: Alexander Scheel <alexander.m.scheel@gmail.com>
* Drop extraneous length check in newMarkupSanitizer(...)
Signed-off-by: Alexander Scheel <alexander.m.scheel@gmail.com>
* Fix plural ELEMENT and ALLOW_ATTR in docs
These were left over from their initial names. Make them singular to
conform with the current expectations.
Signed-off-by: Alexander Scheel <alexander.m.scheel@gmail.com>
* add [ui] Reactions
* move contend check from form to go functions
* use else if
* check if reaction is allowed only on react
(so previous custom reaction can be still removed)
* use $.AllowedReactions in templates
* use ctx.Flash.Error
* use it there too
* add redirection
* back to server error
because a wrong reaction is a template issue ...
* add emoji list link
* add docs entry
* small wording nit
suggestions from @jolheiser - thx
* same reactions as github
* fix PR reactions
* handle error so template JS could check
* Add Integrations Test
* add REACTIONS setting to cheat-sheet doc page
* Add USE_SERVICE_WORKER setting
This will be very useful setting for anyone doing frontend work.
Fixes: https://github.com/go-gitea/gitea/issues/9044
* prevent potential syntax error on old browsers
* Graceful Shutdown for windows and others
Restructures modules/graceful, adding shutdown for windows, removing and
replacing the old minwinsvc code.
Creates a new waitGroup - terminate which allows for goroutines to
finish up after the shutdown of the servers.
Shutdown and terminate hooks are added for goroutines.
* Remove unused functions - these can be added in a different PR
* Add startup timeout functionality
* Document STARTUP_TIMEOUT
