Commit Graph
101 Commits
Author SHA1 Message Date
6543 f337d31455 bump github.com/6543/logfile-open ... (#2240)
to make woodpecker compile on windows again


---
*Sponsored by Kithara Software GmbH*
2023-08-18 15:35:12 +02:00
Harry Pidcock e4ff041882 Improve agent rpc retry logic with exponential backoff (#2205)
Existing retry logic was a simple second delay, replacing it with a
exponential backoff.
Initial delay is 10ms up to 10s for the max delay. In the future this
should be made configurable.

With an extended max delay it becomes important to notice context
cancelation, so this now also selects on both the delay and context
done.
2023-08-18 15:13:13 +02:00
qwerty287 6e0def58a1 Switch to upstream ttlcache (#2187)
We've been using https://github.com/lafriks/ttlcache but it's archived.
It does work with the upstream library too, so its better to use it.
2023-08-10 09:17:12 +02:00
6543 11ba724bbf bump xorm.io to db7c2640627d (#2185)
pull got merged so we can switch back to upstream
2023-08-10 02:39:41 +02:00
6543 db057b8d82 Release file lock on USR1 signal (#2151)
close #2136
2023-08-08 08:47:45 +02:00
6543 17ab945825 Fix 'add-orgs' migration (#2117)
close  #2096

~~blocked by https://gitea.com/xorm/xorm/pulls/2320~~
2023-08-08 00:16:50 +03:00
6543 3d4758578a Add opt save global log output to file (#2115)
close  #1933

---------
*Sponsored by Kithara Software GmbH*
2023-08-07 20:47:30 +02:00
6543 4d2f824fb8 fix docs nits (#2025) 2023-07-21 21:56:24 +02:00
Anbratenand6543 556607b525 Rework log streaming and related functions (#1802)
closes #1801
closes #1815 
closes #1144
closes  #983
closes  #557
closes #1827
regression of #1791

# TODO
- [x] adjust log model
- [x] add migration for logs
- [x] send log line via grpc using step-id
- [x] save log-line to db
- [x] stream log-lines to UI
- [x] use less structs for log-data
- [x] make web UI work
  - [x] display logs loaded from db
  - [x] display streaming logs
- [ ] ~~make migration work~~ -> dedicated pull (#1828)

# TESTED
- [x] new logs are stored in database
- [x] log retrieval via cli (of new logs) works
- [x] log streaming works (tested via curl & webui)
- [x] log retrieval via web (of new logs) works

---------

Co-authored-by: 6543 <6543@obermui.de>
2023-06-06 09:52:08 +02:00
6543andAnbraten d1213afdc8 [Docs] use redocusaurus to display swagger file (#1818)
https://redocusaurus.vercel.app/

followup of  #1782

---------

Co-authored-by: Anbraten <anton@ju60.de>
2023-06-04 05:07:39 +02:00
14177635b6 Update swagger API specification (#1782)
# Summary

This PR drops the outdated former swagger.yaml/json and introduced
automatic API document generation from Go code.
The generated code is also used to generate documentation/markdown for
the community page,
as well as enable the Woodpecker server to serve a Swagger Web UI for
manual tinkering.

I did opt-in for gin-swagger, a middleware for the Gin framework, to
ease implementation and have a sophisticated output.
This middleware only produces Swagger v2 specs. AFAIK the newer OpenApi
3x tooling is not yet that mature,
so I guess that's fine for now.

## Implemenation notes

- former swagger.json files removed
- former // swagger godocs removed
- introduced new dependency gin-swagger, which uses godoc annotations on
top of Gin Handler functions.
- reworked Makefile to automatically generate Go code for the server
- introduce new dependency go-swagger, to generate Markdown for
documentation purposes
- add a Swagger Web UI, incl. capabilities for manual API exploration
- consider relative root paths in the implementation
- write documentation for all exposed API endpoints
- incl. API docs in the community website (auto-generated)
- provide developer documentation, for the Woodpecker authors
- no other existing logic/code was intentionally changed

---------

close #292

---------

Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
Co-authored-by: 6543 <6543@obermui.de>
2023-06-03 21:38:36 +02:00
Patrick Schratzand6543 3d0338315f Revert to docker 20.10.x for API 1.41 compatibility (#1792)
fix #1786

---------

Co-authored-by: 6543 <6543@obermui.de>
2023-06-02 15:09:38 +02:00
qwerty287 dc3f7d61ea Update Gin (#1797)
Fixes the security issue (we aren't affected)

Closes #1756
2023-06-01 08:25:17 +02:00
qwerty287 94e63b43f2 Update github.com/docker/distribution (#1749)
Update github.com/docker/distribution to 2.8.2

https://github.com/distribution/distribution/releases/tag/v2.8.2
https://github.com/advisories/GHSA-hqxw-f8mx-cpmw
2023-05-14 12:48:32 +02:00
qwerty287 2ccf7c6f1a Drop Gogs support (#1752)
Gogs support is broken (and we won't fix it because we don't care about
it...) because it does not support OAuth, at least after we introduced
the new Vue UI.

See:

https://github.com/woodpecker-ci/woodpecker/blob/77d830d5b56f85084a74fe2396689cef8b2cdee2/server/forge/gogs/gogs.go#L84

This route is not present in the new UI.
2023-05-14 10:34:05 +02:00
Anbraten 188123ea74 Update dependencies (#1744) 2023-05-11 00:28:02 +02:00
6543 204d05f447 Implement YAML Map Merge, Overrides, and Sequence Merge Support (#1720)
close  #1192
2023-04-29 14:49:41 +02:00
Lauris BHand6543 46452fbd84 Update Go dependencies and minimal Go version to 1.20 (#1650)
Signed-off-by: 6543 <6543@obermui.de>
Co-authored-by: 6543 <6543@obermui.de>
2023-03-21 00:48:15 +01:00
Lauris BH f1f722645c Update dependencies golang/x libs (#1612) 2023-03-10 19:23:07 +01:00
6543andAnbraten 18d3139e9e Use modern error handling and enforce it via lint (#1327)
Co-authored-by: Anbraten <anton@ju60.de>
2023-02-02 00:08:02 +01:00
6516a28cdd Secrets encryption in database (#1475)
closes #101

Added secrets encryption in database

- Google TINK or simple AES as encryption mechanisms
- Keys rotation support on TINK
- Existing SecretService is wrapped by encryption layer
- Encryption can be enabled and disabled at any time

Co-authored-by: Kuzmin Ilya <ilia.kuzmin@indrive.com>
Co-authored-by: 6543 <6543@obermui.de>
2023-01-12 20:59:07 +01:00
6543 fc4af44b43 router: create apiBase (#1442)
- refactor to dedup string `api`
- bump golang.org/x/next
2022-12-21 16:16:36 +01:00
6543 e8490a757f GenerateScript should not return encoded script (#1397)
followup to #1395
2022-11-06 13:36:34 +01:00
8f183c82a8 Support changed files for Gitea PRs (#1342)
- add tests to fetch changed files
- ignore error if gitea version is to low
- adjust docs accordingly

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
2022-10-28 19:17:30 +02:00
Joonhyeok Ahn (Joon) 186aee61cf Check if repo exists before creating pipeline (#1297)
Close #1257

make sure the repo exists first before triggering the pipeline
2022-10-22 01:34:11 +02:00
qwerty287 38198f83c4 Update all dependencies (#1291) 2022-10-19 10:15:58 +02:00
6543 9fae0dafaa Update dep moby & golang.org/x/text (#1263)
* CVE-2022-36109
* CVE-2022-32149
2022-10-14 13:01:06 +02:00
Anbraten 287800ac62 Add when evaluate filter (#1213)
closes #312 
closes #224
closes #963

Have a look for

https://github.com/antonmedv/expr/blob/master/docs/Language-Definition.md
2022-10-06 01:49:23 +02:00
b4d89a1cce Add ability to trigger manual builds (#1156)
closes #83 
closes #240 

Co-authored-by: Anbraten <anton@ju60.de>
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
Co-authored-by: 6543 <6543@obermui.de>
2022-09-27 11:05:00 +02:00
6543 afb02d2dd5 Update golang.org/x/net dep 2022-09-14 07:32:06 +02:00
3b0263442a Adding initial version of Kubernetes backend (#552)
Co-authored-by: laszlocph <laszlo@laszlo.cloud>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Rynoxx <rynoxx@grid-servers.net>
2022-09-05 06:01:14 +02:00
dbbd369c9a Migrate to certmagic (#360)
closes #219
closes #850

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
2022-09-04 03:24:42 +02:00
383f273392 Add cron feature (#934)
https://woodpecker-ci.org/docs/usage/cron

Co-authored-by: Anbraten <anton@ju60.de>
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
2022-09-01 00:36:32 +02:00
6543 d3eea72663 Bump deps (#1125)
* upgrade to codeberg.org/6543/go-yaml2json v0.2.1

* upgraded github.com/bmatcuk/doublestar/v4 v4.0.2 => v4.2.0

* upgraded github.com/docker/cli v20.10.14+incompatible => v20.10.17+incompatible

* upgraded github.com/docker/docker v20.10.14+incompatible => v20.10.17+incompatible

* upgraded github.com/gin-gonic/gin v1.7.7 => v1.8.1

* upgraded github.com/golang-jwt/jwt/v4 v4.4.1 => v4.4.2

* upgraded github.com/moby/moby v20.10.14+incompatible => v20.10.17+incompatible

* upgraded github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 => v0.0.0-20220808134915-39b0c02b01ae

* upgraded github.com/lafriks/ttlcache/v3 v3.1.0 => v3.2.0

* upgraded github.com/mattn/go-sqlite3 v1.14.12 => v1.14.15

* upgraded github.com/lib/pq v1.10.5 => v1.10.6

* github.com/prometheus/client_golang v1.12.1 => v1.13.0

* upgraded github.com/urfave/cli/v2 v2.5.1 => v2.11.2

* upgraded github.com/rs/zerolog v1.26.1 => v1.27.0

* upgraded golang.org/x/oauth2 v0.0.0-20220411215720-9780585627b5 => v0.0.0-20220822191816-0ebed06d0094

* upgraded github.com/xanzy/go-gitlab v0.64.0 => v0.73.1

* upgraded google.golang.org/grpc v1.47.0 => v1.49.0
2022-08-25 08:09:05 +02:00
Lauris BHandAnbraten 1ac2c42652 Add global and organization secrets (#1027)
* Implement database changes and store methods for global and organization secrets

* Add tests for new store methods
* Add organization secret API and UI
* Add global secrets API and UI

* Add suggestions

* Update warning style

* Apply suggestions from code review

Co-authored-by: Anbraten <anton@ju60.de>

* Fix lint warning

Co-authored-by: Anbraten <anton@ju60.de>
2022-08-14 13:48:53 +02:00
Lauris BH 19dfc331f4 Add method to check organization membership (#1037)
* Add remote method to check organization membership
* Use named return parameters in interface
* Add membership check service
* Update Gitea SDK
2022-07-25 03:09:35 +02:00
6543 69ec44075c Let single line command be a single command (#1009)
* rm go-shlex usage

* update
2022-07-19 07:20:27 +02:00
6543 31bad81979 Use external lib to convert yaml to json (#1028)
this move shared/yml/* into an independent lib
2022-07-17 17:23:31 +02:00
6543 17999da20f Minim golang 1.18 and drop vendor folder (#979) 2022-06-17 01:57:02 +02:00
6543 904f9bb194 Update github.com/containerd/containerd (#978)
* update github.com/containerd/containerd

* go mod tidy && go mod vendor
2022-06-16 17:35:56 +02:00
Anbratenand6543 cc30db44ac Use asym key to sign webhooks (#916)
* use async key pair for webhooks

* fix tests

* fix linter

* improve code

* add key pair to database

* undo some changes

* more undo

* improve docs

* add api-endpoint

* add signaturne api endpoint

* fix error

* fix linting and test

* fix lint

* add test

* migration 006

* no need for migration

* replace httsign lib

* fix lint

Co-authored-by: 6543 <6543@obermui.de>
2022-06-01 20:06:27 +02:00
Anbraten e79ad00826 Add agent tagging / filtering for pipelines (#902)
Officially support labels for pipelines and agents to improve pipeline picking. 

* add pipeline labels
* update, improve docs  and add migration
* update proto file

---
closes #304 & #860
2022-05-31 01:12:18 +02:00
6543 d06dfc86b4 Allow gitea dev version (#914)
* update gitea sdk to latest
* As before try to autodetect gitea version, if this does not work, assume it's latest version (v1.17.0 atm)
2022-05-14 17:34:40 +02:00
6543 1e9119ace9 Update backend dependencies (#898)
* update xorm.io

* update module

* update github.com/docker/cli

* update github.com/docker/distribution

* update github.com/docker/docker

* update github.com/gin-gonic/gin

* update github.com/golang-jwt/jwt/v4

* update github.com/golangci/golangci-lint

* update github.com/gorilla/securecookie

* update github.com/lib/pq

* update github.com/mattn/go-sqlite3

* update github.com/moby/moby

* update github.com/stretchr/testify

* update github.com/urfave/cli/v2

* update github.com/xanzy/go-gitlab

* finish

* update module

* clean
2022-05-05 19:36:49 +02:00
qwerty287 9c6c4559a7 Add SSH backend (#861)
Add SSH backend that runs commands via SSH.

Close #848
2022-04-29 12:30:50 +02:00
Lukas BachschwellandAnbraten 59ba8538a1 Add support for pipeline configuration service (#804)
* Add configuration extension flags to server
Add httpsignatures dependency

Signed-off-by: Lukas Bachschwell <lukas@lbsfilm.at>

* Add http fetching to config fetcher

Signed-off-by: Lukas Bachschwell <lukas@lbsfilm.at>

* Refetch config on rebuild

Signed-off-by: Lukas Bachschwell <lukas@lbsfilm.at>

* - Ensure multipipeline compatiblity
- Send original config in http request

Signed-off-by: Lukas Bachschwell <lukas@lbsfilm.at>

* Basic tests of config api

Signed-off-by: Lukas Bachschwell <lukas@lbsfilm.at>

* Simple docs page

Signed-off-by: Lukas Bachschwell <lukas@lbsfilm.at>

* Better flag naming

Signed-off-by: Lukas Bachschwell <lukas@lbsfilm.at>

* Rename usages of the term yaml
Rename ConfigAPI struct

Signed-off-by: Lukas Bachschwell <lukas@lbsfilm.at>

* Doc adjustments

Signed-off-by: Lukas Bachschwell <lukas@lbsfilm.at>

* More docs touchups

Signed-off-by: Lukas Bachschwell <lukas@lbsfilm.at>

* Fix env vars in docs

Signed-off-by: Lukas Bachschwell <lukas@lbsfilm.at>

* fix json tags for api calls

Signed-off-by: Lukas Bachschwell <lukas@lbsfilm.at>

* Add example config service

Signed-off-by: Lukas Bachschwell <lukas@lbsfilm.at>

* Consistent naming for configService

Signed-off-by: Lukas Bachschwell <lukas@lbsfilm.at>

* Docs: Change example repository location

Signed-off-by: Lukas Bachschwell <lukas@lbsfilm.at>

* Fix tests after response field rename

Signed-off-by: Lukas Bachschwell <lukas@lbsfilm.at>

* Revert accidential unrelated change in api hook

Signed-off-by: Lukas Bachschwell <lukas@lbsfilm.at>

* Update server flag descriptions

Co-authored-by: Anbraten <anton@ju60.de>

Co-authored-by: Anbraten <anton@ju60.de>
2022-02-28 10:56:23 +01:00
6543 56a854fe14 Update deps (#789)
* update github.com/docker/cli

* update github.com/docker/distribution

* update github.com/docker/docker

* update github.com/gin-gonic/gin

* update github.com/golang-jwt/jwt/v4

* update github.com/golangci/golangci-lint

* update github.com/gorilla/securecookie

* update github.com/mattn/go-sqlite3

* update github.com/moby/moby

* update github.com/prometheus/client_golang

* update github.com/xanzy/go-gitlab
2022-02-24 17:33:24 +01:00
6543 b6e47a3f4a Update deps (#724)
* update github.com/containerd/containerd v1.5.7 -> v1.5.9

* update github.com/lib/pq v1.10.3 -> v1.10.4

* update github.com/prometheus/client_golang v1.11.0 -> v1.12.0

* update github.com/rs/zerolog v1.25.0 -> v1.26.1

* update golang.org/x/crypto 2021-12-15 -> 2022-01-28
2022-01-29 16:04:50 +01:00
6543 70fcc173b9 Update github.com/xanzy/go-gitlab v0.51.1 -> v0.52.2 (#599) 2021-12-12 16:39:25 +01:00
6543 0061edcbe2 Remove gopkg.in/yaml.v2 (#583)
* rm "gopkg.in/yaml.v2"

* fix UnmarshalYAML for Networks & Ulimits
2021-12-08 23:35:51 +01:00