mirror of
https://github.com/goreleaser/goreleaser.git
synced 2025-03-21 21:07:19 +02:00
f6615b138b
This PR corrects grammar mistakes in documentation: - artefacts -> artifacts - Dockefile -> Dockerfile - compliation -> compilation - gorleaser -> goreleaser - repositores -> repositories - succesfull -> successful
317 lines
8.9 KiB
Markdown
317 lines
8.9 KiB
Markdown
---
|
|
date: 2022-02-05
|
|
slug: cloud-native-storage
|
|
categories:
|
|
- tutorials
|
|
authors:
|
|
- dirien
|
|
---
|
|
|
|
# How to use GoReleaser with Cloud Native Storage
|
|
|
|
In this tutorial, I want to describe, how quickly we can deploy our release
|
|
artifacts to a cloud native storage when using GoReleaser.
|
|
It’s just a few additional lines in your `.goreleaser.yaml`.
|
|
|
|
<!-- more -->
|
|
|
|
To better show this, I created a little demo and use the storage services of the
|
|
big three cloud providers: Azure Blob Storage, AWS S3 and Google Cloud Storage.
|
|
|
|
|
|
|
|
You can use any S3 compatible storage provider too.
|
|
**GoReleaser** support this too! The most prominent (self-hosted) solution is
|
|
**MinIO**.
|
|
|
|
|
|
|
|
## The infrastructure code
|
|
|
|
I created a very simple **Terraform** deployment to provision on all three cloud
|
|
provider their appropriate cloud storage service.
|
|
It’s a demo, why not?
|
|
|
|
You don’t need to use **Terraform** for this, you could use any other means like
|
|
**Pulumi**, **CLI** or even the **UI**.
|
|
|
|
###### `main.tf`
|
|
|
|
```terraform
|
|
terraform {
|
|
required_providers {
|
|
google = {
|
|
source = "hashicorp/google"
|
|
version = "4.9.0"
|
|
}
|
|
azurerm = {
|
|
source = "hashicorp/azurerm"
|
|
version = "2.94.0"
|
|
}
|
|
aws = {
|
|
source = "hashicorp/aws"
|
|
version = "3.74.0"
|
|
}
|
|
}
|
|
}
|
|
|
|
provider "azurerm" {
|
|
features {}
|
|
}
|
|
|
|
provider "google" {
|
|
credentials = file(var.gcp_auth_file)
|
|
project = var.gcp_project
|
|
region = var.gcp_region
|
|
}
|
|
|
|
provider "aws" {
|
|
region = var.aws_region
|
|
}
|
|
```
|
|
|
|
###### `variables.tf`
|
|
|
|
```terraform
|
|
variable "gcp_project" {
|
|
type = string
|
|
}
|
|
|
|
variable "gcp_region" {
|
|
default = "europe-west6"
|
|
}
|
|
|
|
variable "gcp_zone" {
|
|
default = "europe-west6-a"
|
|
}
|
|
|
|
variable "gcp_bucket_location" {
|
|
default = "EU"
|
|
}
|
|
|
|
variable "gcp_auth_file" {
|
|
default = "./auth.json"
|
|
description = "Path to the GCP auth file"
|
|
}
|
|
|
|
variable "aws_region" {
|
|
default = "eu-central-1"
|
|
}
|
|
|
|
variable "azure_location" {
|
|
default = "West Europe"
|
|
}
|
|
|
|
variable "name" {
|
|
default = "goreleaser-quickbites"
|
|
}
|
|
```
|
|
|
|
###### `blob.tf`
|
|
|
|
```terraform
|
|
resource "google_storage_bucket" "goreleaser-gcp-storage-bucket" {
|
|
name = var.name
|
|
location = var.gcp_bucket_location
|
|
force_destroy = true
|
|
uniform_bucket_level_access = false
|
|
}
|
|
resource "google_storage_bucket_access_control" "goreleaser-gcp-storage-bucket-access-control" {
|
|
bucket = google_storage_bucket.goreleaser-gcp-storage-bucket.name
|
|
role = "READER"
|
|
entity = "allUsers"
|
|
}
|
|
|
|
resource "azurerm_resource_group" "goreleaser-azure-resource-group" {
|
|
name = var.name
|
|
location = var.azure_location
|
|
}
|
|
|
|
resource "azurerm_storage_account" "goreleaser-azure-storage-account" {
|
|
name = "gorleaserquickbites"
|
|
resource_group_name = azurerm_resource_group.goreleaser-azure-resource-group.name
|
|
location = azurerm_resource_group.goreleaser-azure-resource-group.location
|
|
account_tier = "Standard"
|
|
account_replication_type = "LRS"
|
|
allow_blob_public_access = true
|
|
network_rules {
|
|
default_action = "Allow"
|
|
}
|
|
}
|
|
|
|
resource "azurerm_storage_container" "goreleaser-storage-container" {
|
|
name = var.name
|
|
storage_account_name = azurerm_storage_account.goreleaser-azure-storage-account.name
|
|
container_access_type = "container"
|
|
}
|
|
|
|
resource "aws_s3_bucket" "goreleaser-s3-bucket" {
|
|
bucket = var.name
|
|
acl = "public-read"
|
|
}
|
|
```
|
|
|
|
###### Apply the Terraform script:
|
|
|
|
```bash
|
|
terraform apply -var "gcp_project=xxx"
|
|
```
|
|
|
|
```
|
|
...
|
|
azurerm_storage_container.goreleaser-storage-container: Creation complete after 0s [id=https://goreleaserquickbites.blob.core.windows.net/goreleaser-quickbites]
|
|
|
|
Apply complete! Resources: 6 added, 0 changed, 0 destroyed.
|
|
|
|
Outputs:
|
|
|
|
aws-s3-bucket-name = "goreleaser-quickbites"
|
|
azure-storage-account-key = <sensitive>
|
|
azure-storage-account-name = "export AZURE_STORAGE_ACCOUNT=goreleaserquickbites"
|
|
gcp-bucket-url = "gs://goreleaser-quickbites"
|
|
```
|
|
|
|
###### Run this command
|
|
|
|
```bash
|
|
terraform output azure-storage-account-key
|
|
```
|
|
|
|
to get the Azure Storage Account Key, as it is a output field with sensitive data in it.
|
|
|
|
```bash
|
|
export AZURE_STORAGE_KEY=xxxx
|
|
```
|
|
|
|
Now we can add in our `.goreleaser.yaml` the new **blobs** field.
|
|
Important is here to set the right provider: **gs** (for Google Cloud Storage),
|
|
**azblob** (for Azure Blob) and **s3** (for AWS S3 or compatible provider)!
|
|
|
|
```yaml
|
|
# This is an example .goreleaser.yml file with some sensible defaults.
|
|
# Make sure to check the documentation at https://goreleaser.com
|
|
before:
|
|
hooks:
|
|
- go mod tidy
|
|
builds:
|
|
- env:
|
|
- CGO_ENABLED=0
|
|
goos:
|
|
- linux
|
|
- darwin
|
|
|
|
release:
|
|
disable: true
|
|
---
|
|
blobs:
|
|
- provider: gs
|
|
bucket: goreleaser-quickbites
|
|
- provider: azblob
|
|
bucket: goreleaser-quickbites
|
|
- provider: s3
|
|
bucket: goreleaser-quickbites
|
|
region: eu-central-1
|
|
```
|
|
|
|
In this demo, I disabled the **release **section, as I don’t want to upload to
|
|
GitHub.
|
|
|
|
## Authentication
|
|
|
|
In terms of authentication the GoReleaser’s blob pipe authentication varies depending upon the blob provider as mentioned below:
|
|
|
|
### S3 Provider
|
|
|
|
S3 provider support AWS [default credential
|
|
provider](https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html#specifying-credentials)
|
|
chain in the following order:
|
|
|
|
- Environment variables.
|
|
- Shared credentials file.
|
|
- If your application is running on an Amazon EC2 instance, IAM role for Amazon EC2.
|
|
|
|
### Azure Blob Provider Currently it supports authentication only
|
|
|
|
with [environment variables](https://docs.microsoft.com/en-us/azure/storage/common/storage-azure-cli#set-default-azure-storage-account-environment-variables):
|
|
|
|
- AZURE_STORAGE_ACCOUNT
|
|
- AZURE_STORAGE_KEY or AZURE_STORAGE_SAS_TOKEN
|
|
|
|
### GCS
|
|
|
|
Provider GCS provider uses [Application Default
|
|
Credentials](https://cloud.google.com/docs/authentication/production) in the
|
|
following order:
|
|
|
|
- Environment Variable (GOOGLE_APPLICATION_CREDENTIALS)
|
|
- Default Service Account from the compute instance (Compute Engine, Kubernetes
|
|
Engine, Cloud function etc).
|
|
|
|
## Run GoReleaser
|
|
|
|
After configuring we can finally execute **GoReleaser**, in your pipeline code
|
|
via the command:
|
|
|
|
```bash
|
|
goreleaser release --rm-dist
|
|
```
|
|
|
|
If everything went smooth, you should see a similar output, showing the upload of your artifacts.
|
|
|
|
```
|
|
...
|
|
• publishing
|
|
• blobs
|
|
• uploading path=quick-bites/0.1/quick-bites_0.1_checksums.txt
|
|
• uploading path=quick-bites/0.1/quick-bites_0.1_darwin_amd64.tar.gz
|
|
• uploading path=quick-bites/0.1/quick-bites_0.1_linux_arm64.tar.gz
|
|
• uploading path=quick-bites/0.1/quick-bites_0.1_darwin_arm64.tar.gz
|
|
• uploading path=quick-bites/0.1/quick-bites_0.1_linux_amd64.tar.gz
|
|
• uploading path=quick-bites/0.1/quick-bites_0.1_linux_386.tar.gz
|
|
• uploading path=quick-bites/0.1/quick-bites_0.1_checksums.txt
|
|
• uploading path=quick-bites/0.1/quick-bites_0.1_checksums.txt
|
|
• uploading path=quick-bites/0.1/quick-bites_0.1_linux_386.tar.gz
|
|
• uploading path=quick-bites/0.1/quick-bites_0.1_linux_amd64.tar.gz
|
|
• uploading path=quick-bites/0.1/quick-bites_0.1_linux_arm64.tar.gz
|
|
• uploading path=quick-bites/0.1/quick-bites_0.1_linux_amd64.tar.gz
|
|
• uploading path=quick-bites/0.1/quick-bites_0.1_darwin_amd64.tar.gz
|
|
• uploading path=quick-bites/0.1/quick-bites_0.1_darwin_arm64.tar.gz
|
|
• uploading path=quick-bites/0.1/quick-bites_0.1_linux_386.tar.gz
|
|
• uploading path=quick-bites/0.1/quick-bites_0.1_linux_arm64.tar.gz
|
|
• uploading path=quick-bites/0.1/quick-bites_0.1_darwin_arm64.tar.gz
|
|
• uploading path=quick-bites/0.1/quick-bites_0.1_darwin_amd64.tar.gz
|
|
• release succeeded after 22.63s
|
|
...
|
|
```
|
|
|
|
> One note: The provider fails silently, if your credentials are wrong. You
|
|
> would still see uploading and release succeeded. Keep this in mind, if the
|
|
> files are not appearing in the UI. I wasted some time on this. The culprit is
|
|
> the underlying library GoReleaser is using.
|
|
|
|
Let’s check in the consoles of the cloud provider too, If the files are present.
|
|
|
|
###### Google Cloud Storage:
|
|
|
|
|
|
|
|
###### Azure Blob Storage
|
|
|
|
|
|
|
|
###### AWS S3
|
|
|
|
|
|
|
|
Looks very good! Now you can share the URLs of the files for further use!
|
|
|
|
## Want more Informations?
|
|
|
|
If you want to know more about some advanced options, feel free to check out the
|
|
[official documentation about the blob support in
|
|
GoReleaser](https://goreleaser.com/customization/blob/)
|
|
|
|
And here is the example code: [dirien/quick-bytes](https://github.com/dirien/quick-bites/tree/main/goreleaser-blob)
|
|
|
|
|