foster/forgejo
1
0
Fork 0
mirror of https://codeberg.org/forgejo/forgejo.git synced 2024-11-21 08:31:27 -05:00
Code Issues Wiki Activity

Add grafana gitea-mixin (#17758)

Browse source 
This PR adds gitea-mixin, configurable Grafana dashboards (and potentially prometheus alerts+recording rules) based on Gitea [metrics](https://docs.gitea.io/en-us/config-cheat-sheet/#metrics-metrics).

The overview dashboard is described using jsonnet and grafonnet library: https://grafana.github.io/grafonnet-lib/

Mixins help to define dashboard and alerts as code so they can be collaboratively improved by the users.

![image](https://user-images.githubusercontent.com/14870891/142862822-fe57b384-c74a-4103-8548-033e92f90751.png)

__
## Generate config files

You can manually generate dashboards, but first you should install some tools:

```bash
go get github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb
go get github.com/google/go-jsonnet/cmd/jsonnet
# or in brew: brew install go-jsonnet
```

For linting and formatting, you would also need `mixtool` and `jsonnetfmt` installed. If you
have a working Go development environment, it's easiest to run the following:

```bash
go get github.com/monitoring-mixins/mixtool/cmd/mixtool
go get github.com/google/go-jsonnet/cmd/jsonnetfmt
```

The files in `dashboards_out` need to be imported
into your Grafana server.  The exact details will be depending on your environment.

Edit `config.libsonnet` (for example, list of Gitea metrics to be shown under stats can be adjusted). if required and then build JSON dashboard files for Grafana:

```bash
make
```


For more about mixins, please see:
https://github.com/monitoring-mixins/docs
https://www.youtube.com/watch?v=GDdnL5R_l-Y* add gitea mixin

* remove alerts/rules

* gitea-mixin: add interval factor of 1/2 to remove duplicated change events

* gitea-mixin: fix changes panel, add aggregation interval for changes panel

* gitea-mixin: add totals singlestat

* gitea mixin: switch change graph to timeseries type

* add color overrides for issue labels

* bump grafonnet version

* gitea-mixin: convert graphs to timeseries

* gitea-mixin:  make fmt

* gitea-mixin: add .PHONE in Makefile

* gitea-mixin: add time configration

* gitea-mixin: make fmt and collapse addPanel grid

* gitea-mixin: add static ids for shared panels

* gitea-mixin: add flags showIssuesByRepository, showIssuesByLabel to show/hide corresponding panels

* gitea-mixin: update aggregation interval

* gitea-mixin: update defaults

* gitea-mixin: update panel names

* rename dir to gitea-monitoring-mixin

* gitea-mixin: add gitea_issues_open, gitea_issues_closed metrics

* gitea-mixin: update visible name for datasource

* gitea-mixin: update README

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
This commit is contained in:
Vitaly Zhuravlev 2021-12-20 10:52:33 +03:00 committed by GitHub
parent ff2fd08228
commit 2cd1479e77
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
12 changed files with 668 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
contrib/gitea-monitoring-mixin/.gitignore vendored Normal file
View file

@ -0,0 +1,2 @@
dashboards_out
vendor

31
contrib/gitea-monitoring-mixin/Makefile Normal file
View file

@ -0,0 +1,31 @@
JSONNET_FMT := jsonnetfmt -n 2 --max-blank-lines 1 --string-style s --comment-style s
.PHONY: all
all: build dashboards_out
vendor: jsonnetfile.json
jb install
.PHONY: build
build: vendor
.PHONY: fmt
fmt:
find . -name 'vendor' -prune -o -name '*.libsonnet' -print -o -name '*.jsonnet' -print | \
xargs -n 1 -- $(JSONNET_FMT) -i
.PHONY: lint
lint: build
find . -name 'vendor' -prune -o -name '*.libsonnet' -print -o -name '*.jsonnet' -print | \
while read f; do \
$(JSONNET_FMT) "$$f" | diff -u "$$f" -; \
done
mixtool lint mixin.libsonnet
dashboards_out: mixin.libsonnet config.libsonnet $(wildcard dashboards/*)
@mkdir -p dashboards_out
jsonnet -J vendor -m dashboards_out lib/dashboards.jsonnet
.PHONY: clean
clean:
rm -rf dashboards_out

33
contrib/gitea-monitoring-mixin/README.md Normal file
View file

@ -0,0 +1,33 @@
# Gitea Mixin
Gitea Mixin is a set of configurable Grafana dashboards based on the metrics exported by the Gitea built-in metrics endpoint.
## Generate config files
You can manually generate dashboards, but first you should install some tools:
```bash
go install github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb@latest
go install github.com/google/go-jsonnet/cmd/jsonnet@latest
# or in brew: brew install go-jsonnet
```
For linting and formatting, you would also need `mixtool` and `jsonnetfmt` installed. If you
have a working Go development environment, it's easiest to run the following:
```bash
go install github.com/monitoring-mixins/mixtool/cmd/mixtool@latest
go install github.com/google/go-jsonnet/cmd/jsonnetfmt@latest
```
The files in `dashboards_out` need to be imported
into your Grafana server. The exact details will be depending on your environment.
Edit `config.libsonnet` if required and then build JSON dashboard files for Grafana:
```bash
make
```
For more advanced uses of mixins, see
https://github.com/monitoring-mixins/docs.

99
contrib/gitea-monitoring-mixin/config.libsonnet Normal file
View file

@ -0,0 +1,99 @@
{
_config+:: {
local c = self,
dashboardNamePrefix: 'Gitea',
dashboardTags: ['gitea'],
dashboardPeriod: 'now-1h',
dashboardTimezone: 'default',
dashboardRefresh: '1m',
// please see https://docs.gitea.io/en-us/config-cheat-sheet/#metrics-metrics
// Show issue by repository metrics with format gitea_issues_by_repository{repository="org/repo"} 5.
// Requires Gitea 1.16.0 with ENABLED_ISSUE_BY_REPOSITORY set to true.
showIssuesByRepository: true,
// Show graphs for issue by label metrics with format gitea_issues_by_label{label="bug"} 2.
// Requires Gitea 1.16.0 with ENABLED_ISSUE_BY_LABEL set to true.
showIssuesByLabel: true,
// Requires Gitea 1.16.0.
showIssuesOpenClose: true,
// add or remove metrics from dashboard
giteaStatMetrics:
[
{
name: 'gitea_organizations',
description: 'Organizations',
},
{
name: 'gitea_teams',
description: 'Teams',
},
{
name: 'gitea_users',
description: 'Users',
},
{
name: 'gitea_repositories',
description: 'Repositories',
},
{
name: 'gitea_milestones',
description: 'Milestones',
},
{
name: 'gitea_stars',
description: 'Stars',
},
{
name: 'gitea_releases',
description: 'Releases',
},
]
+
if c.showIssuesOpenClose then
[
{
name: 'gitea_issues_open',
description: 'Issues opened',
},
{
name: 'gitea_issues_closed',
description: 'Issues closed',
},
] else
[
{
name: 'gitea_issues',
description: 'Issues',
},
],
//set this for using label colors on graphs
issueLabels: [
{
label: 'bug',
color: '#ee0701',
},
{
label: 'duplicate',
color: '#cccccc',
},
{
label: 'invalid',
color: '#e6e6e6',
},
{
label: 'enhancement',
color: '#84b6eb',
},
{
label: 'help wanted',
color: '#128a0c',
},
{
label: 'question',
color: '#cc317c',
},
],
},
}

1
contrib/gitea-monitoring-mixin/dashboards/dashboards.libsonnet Normal file
View file

@ -0,0 +1 @@
(import 'overview.libsonnet')

461
contrib/gitea-monitoring-mixin/dashboards/overview.libsonnet Normal file
View file

@ -0,0 +1,461 @@
local grafana = import 'github.com/grafana/grafonnet-lib/grafonnet/grafana.libsonnet';
local prometheus = grafana.prometheus;
local addIssueLabelsOverrides(labels) =
{
fieldConfig+: {
overrides+: [
{
matcher: {
id: 'byRegexp',
options: label.label,
},
properties: [
{
id: 'color',
value: {
fixedColor: label.color,
mode: 'fixed',
},
},
],
}
for label in labels
],
},
};
{
grafanaDashboards+:: {
local giteaSelector = 'job="$job", instance="$instance"',
local giteaStatsPanel =
grafana.statPanel.new(
'Gitea stats',
datasource='$datasource',
reducerFunction='lastNotNull',
graphMode='none',
colorMode='value',
)
.addTargets(
[
prometheus.target(expr='%s{%s}' % [metric.name, giteaSelector], legendFormat=metric.description, intervalFactor=10)
for metric in $._config.giteaStatMetrics
]
)
+ {
fieldConfig+: {
defaults+: {
color: {
fixedColor: 'blue',
mode: 'fixed',
},
},
},
},
local giteaUptimePanel =
grafana.statPanel.new(
'Uptime',
datasource='$datasource',
reducerFunction='last',
graphMode='area',
colorMode='value',
)
.addTarget(prometheus.target(expr='time()-process_start_time_seconds{%s}' % giteaSelector, intervalFactor=1))
+ {
fieldConfig+: {
defaults+: {
color: {
fixedColor: 'blue',
mode: 'fixed',
},
unit: 's',
},
},
},
local giteaMemoryPanel =
grafana.graphPanel.new(
'Memory usage',
datasource='$datasource'
)
.addTarget(prometheus.target(expr='process_resident_memory_bytes{%s}' % giteaSelector, intervalFactor=2))
+ {
type: 'timeseries',
options+: {
tooltip: {
mode: 'multi',
},
legend+: {
displayMode: 'hidden',
},
},
fieldConfig+: {
defaults+: {
custom+: {
lineInterpolation: 'smooth',
fillOpacity: 15,
},
color: {
fixedColor: 'green',
mode: 'fixed',
},
unit: 'decbytes',
},
},
},
local giteaCpuPanel =
grafana.graphPanel.new(
'CPU usage',
datasource='$datasource'
)
.addTarget(prometheus.target(expr='rate(process_cpu_seconds_total{%s}[$__rate_interval])*100' % giteaSelector, intervalFactor=2))
+ {
type: 'timeseries',
options+: {
tooltip: {
mode: 'multi',
},
legend+: {
displayMode: 'hidden',
},
},
fieldConfig+: {
defaults+: {
custom+: {
lineInterpolation: 'smooth',
gradientMode: 'scheme',
fillOpacity: 15,
axisSoftMin: 0,
axisSoftMax: 0,
},
color: {
mode: 'continuous-GrYlRd', // from green to red (100%)
},
unit: 'percent',
},
overrides: [
{
matcher: {
id: 'byRegexp',
options: '.+',
},
properties: [
{
id: 'max',
value: 100,
},
{
id: 'min',
value: 0,
},
],
},
],
},
},
local giteaFileDescriptorsPanel =
grafana.graphPanel.new(
'File descriptors usage',
datasource='$datasource',
)
.addTarget(prometheus.target(expr='process_open_fds{%s}' % giteaSelector, intervalFactor=2))
.addTarget(prometheus.target(expr='process_max_fds{%s}' % giteaSelector, intervalFactor=2))
.addSeriesOverride(
{
alias: '/process_max_fds.+/',
color: '#F2495C', // red
dashes: true,
fill: 0,
},
)
+ {
type: 'timeseries',
options+: {
tooltip: {
mode: 'multi',
},
legend+: {
displayMode: 'hidden',
},
},
fieldConfig+: {
defaults+: {
custom+: {
lineInterpolation: 'smooth',
gradientMode: 'scheme',
fillOpacity: 0,
},
color: {
fixedColor: 'green',
mode: 'fixed',
},
unit: '',
},
overrides: [
{
matcher: {
id: 'byFrameRefID',
options: 'B',
},
properties: [
{
id: 'custom.lineStyle',
value: {
fill: 'dash',
dash: [
10,
10,
],
},
},
{
id: 'color',
value: {
mode: 'fixed',
fixedColor: 'red',
},
},
],
},
],
},
},
local giteaChangesPanelPrototype =
grafana.graphPanel.new(
'',
datasource='$datasource',
interval='$agg_interval',
maxDataPoints=10000,
)
+ {
type: 'timeseries',
options+: {
tooltip: {
mode: 'multi',
},
legend+: {
calcs+: [
'sum',
],
},
},
fieldConfig+: {
defaults+: {
noValue: '0',
custom+: {
drawStyle: 'bars',
barAlignment: -1,
fillOpacity: 50,
gradientMode: 'hue',
pointSize: 1,
lineWidth: 0,
stacking: {
group: 'A',
mode: 'normal',
},
},
},
},
},
local giteaChangesPanelAll =
giteaChangesPanelPrototype
.addTarget(prometheus.target(expr='changes(process_start_time_seconds{%s}[$__interval]) > 0' % [giteaSelector], legendFormat='Restarts', intervalFactor=1))
.addTargets(
[
prometheus.target(expr='floor(delta(%s{%s}[$__interval])) > 0' % [metric.name, giteaSelector], legendFormat=metric.description, intervalFactor=1)
for metric in $._config.giteaStatMetrics
]
) + { id: 200 }, // some unique number, beyond the maximum number of panels in the dashboard,
local giteaChangesPanelTotal =
grafana.statPanel.new(
'Changes',
datasource='-- Dashboard --',
reducerFunction='sum',
graphMode='none',
textMode='value_and_name',
colorMode='value',
)
+ {
targets+: [
{
panelId: giteaChangesPanelAll.id,
refId: 'A',
},
],
}
+ {
fieldConfig+: {
defaults+: {
color: {
mode: 'palette-classic',
},
},
},
},
local giteaChangesByRepositories =
giteaChangesPanelPrototype
.addTarget(prometheus.target(expr='floor(increase(gitea_issues_by_repository{%s}[$__interval])) > 0' % [giteaSelector], legendFormat='{{ repository }}', intervalFactor=1))
+ { id: 210 }, // some unique number, beyond the maximum number of panels in the dashboard,
local giteaChangesByRepositoriesTotal =
grafana.statPanel.new(
'Issues by repository',
datasource='-- Dashboard --',
reducerFunction='sum',
graphMode='none',
textMode='value_and_name',
colorMode='value',
)
+ {
id: 211,
targets+: [
{
panelId: giteaChangesByRepositories.id,
refId: 'A',
},
],
}
+ {
fieldConfig+: {
defaults+: {
color: {
mode: 'palette-classic',
},
},
},
},
local giteaChangesByLabel =
giteaChangesPanelPrototype
.addTarget(prometheus.target(expr='floor(increase(gitea_issues_by_label{%s}[$__interval])) > 0' % [giteaSelector], legendFormat='{{ label }}', intervalFactor=1))
+ addIssueLabelsOverrides($._config.issueLabels)
+ { id: 220 }, // some unique number, beyond the maximum number of panels in the dashboard,
local giteaChangesByLabelTotal =
grafana.statPanel.new(
'Issues by labels',
datasource='-- Dashboard --',
reducerFunction='sum',
graphMode='none',
textMode='value_and_name',
colorMode='value',
)
+ addIssueLabelsOverrides($._config.issueLabels)
+ {
id: 221,
targets+: [
{
panelId: giteaChangesByLabel.id,
refId: 'A',
},
],
}
+ {
fieldConfig+: {
defaults+: {
color: {
mode: 'palette-classic',
},
},
},
},
'gitea-overview.json':
grafana.dashboard.new(
'%s Overview' % $._config.dashboardNamePrefix,
time_from='%s' % $._config.dashboardPeriod,
editable=false,
tags=($._config.dashboardTags),
timezone='%s' % $._config.dashboardTimezone,
refresh='%s' % $._config.dashboardRefresh,
graphTooltip='shared_crosshair',
uid='gitea-overview'
)
.addTemplate(
{
current: {
text: 'Prometheus',
value: 'Prometheus',
},
hide: 0,
label: 'Data Source',
name: 'datasource',
options: [],
query: 'prometheus',
refresh: 1,
regex: '',
type: 'datasource',
},
)
.addTemplate(
{
hide: 0,
label: null,
name: 'job',
options: [],
query: 'label_values(gitea_organizations, job)',
refresh: 1,
regex: '',
type: 'query',
},
)
.addTemplate(
{
hide: 0,
label: null,
name: 'instance',
options: [],
query: 'label_values(gitea_organizations{job="$job"}, instance)',
refresh: 1,
regex: '',
type: 'query',
},
)
.addTemplate(
{
hide: 0,
label: 'aggregation interval',
name: 'agg_interval',
auto_min: '1m',
auto: true,
query: '1m,10m,1h,1d,7d',
type: 'interval',
},
)
.addPanel(grafana.row.new(title='General'), gridPos={ x: 0, y: 0, w: 0, h: 0 },)
.addPanel(giteaStatsPanel, gridPos={ x: 0, y: 0, w: 16, h: 4 })
.addPanel(giteaUptimePanel, gridPos={ x: 16, y: 0, w: 8, h: 4 })
.addPanel(giteaMemoryPanel, gridPos={ x: 0, y: 4, w: 8, h: 6 })
.addPanel(giteaCpuPanel, gridPos={ x: 8, y: 4, w: 8, h: 6 })
.addPanel(giteaFileDescriptorsPanel, gridPos={ x: 16, y: 4, w: 8, h: 6 })
.addPanel(grafana.row.new(title='Changes', collapse=false), gridPos={ x: 0, y: 10, w: 24, h: 8 })
.addPanel(giteaChangesPanelTotal, gridPos={ x: 0, y: 12, w: 6, h: 8 })
+ // use patching instead of .addPanel() to keep static ids
{
panels+: std.flattenArrays([
[
giteaChangesPanelAll { gridPos: { x: 6, y: 12, w: 18, h: 8 } },
],
if $._config.showIssuesByRepository then
[
giteaChangesByRepositoriesTotal { gridPos: { x: 0, y: 20, w: 6, h: 8 } },
giteaChangesByRepositories { gridPos: { x: 6, y: 20, w: 18, h: 8 } },
] else [],
if $._config.showIssuesByLabel then
[
giteaChangesByLabelTotal { gridPos: { x: 0, y: 28, w: 6, h: 8 } },
giteaChangesByLabel { gridPos: { x: 6, y: 28, w: 18, h: 8 } },
] else [],
]),
},
},
}

15
contrib/gitea-monitoring-mixin/jsonnetfile.json Normal file
View file

@ -0,0 +1,15 @@
{
"version": 1,
"dependencies": [
{
"source": {
"git": {
"remote": "https://github.com/grafana/grafonnet-lib.git",
"subdir": "grafonnet"
}
},
"version": "master"
}
],
"legacyImports": false
}

16
contrib/gitea-monitoring-mixin/jsonnetfile.lock.json Normal file
View file

@ -0,0 +1,16 @@
{
"version": 1,
"dependencies": [
{
"source": {
"git": {
"remote": "https://github.com/grafana/grafonnet-lib.git",
"subdir": "grafonnet"
}
},
"version": "3626fc4dc2326931c530861ac5bebe39444f6cbf",
"sum": "gF8foHByYcB25jcUOBqP6jxk0OPifQMjPvKY0HaCk6w="
}
],
"legacyImports": false
}

1
contrib/gitea-monitoring-mixin/lib/alerts.jsonnet Normal file
View file

@ -0,0 +1 @@
std.manifestYamlDoc((import '../mixin.libsonnet').prometheusAlerts)

6
contrib/gitea-monitoring-mixin/lib/dashboards.jsonnet Normal file
View file

@ -0,0 +1,6 @@
local dashboards = (import '../mixin.libsonnet').grafanaDashboards;
{
[name]: dashboards[name]
for name in std.objectFields(dashboards)
}

1
contrib/gitea-monitoring-mixin/lib/rules.jsonnet Normal file
View file

@ -0,0 +1 @@
std.manifestYamlDoc((import '../mixin.libsonnet').prometheusRules)

2
contrib/gitea-monitoring-mixin/mixin.libsonnet Normal file
View file

@ -0,0 +1,2 @@
(import 'dashboards/dashboards.libsonnet') +
(import 'config.libsonnet')