go/golang-saas-starter-kit
1
0
Fork 0
mirror of https://github.com/raseels-repos/golang-saas-starter-kit.git synced 2025-07-03 00:58:13 +02:00
Code Issues Releases Activity

Merge branch 'master' of gitlab.com:geeks-accelerator/oss/saas-starter-kit

Browse Source
This commit is contained in:
Lucas Brown
2019-07-13 19:12:26 -08:00
parent 0d9e195a3b 6607d719d9
commit 83d56497df
7 changed files with 405 additions and 96 deletions
Download Patch File Download Diff File Expand all files Collapse all files

175
README.md
View File

@ -422,6 +422,181 @@ For additional details refer to https://jmoiron.github.io/sqlx/#bindvars
## Contribute
### Development Notes regarding this copy of the project.
#### GitLab CI / CD
_Shared Runners_ have been disabled for this project. Since the project is open source and we wanted to avoid putting
our AWS credentials as pipeline variables. Instead we have deployed our own set of autoscaling runnings on AWS EC2 that
utilize AWS IAM Roles. All other configure is defined for CI/CD is defined in `.gitlab-ci.yaml`.
Below outlines the basic steps to setup [Autoscaling GitLab Runner on AWS](https://docs.gitlab.com/runner/configuration/runner_autoscale_aws/).
1. Define an [AWS IAM Role](https://console.aws.amazon.com/iam/home?region=us-west-2#/roles$new?step=type) that will be
attached to the GitLab Runner instances. The role will need permission to scale (EC2), update the cache (via S3) and
perform the project specific deployment commands.
```
Trusted Entity: AWS Service
Service that will use this role: EC2
Attach permissions policies: AmazonEC2FullAccess, AmazonS3FullAccess, saas-starter-kit-deploy
Role Name: SaasStarterKitEc2RoleForGitLabRunner
Role Description: Allows GitLab runners hosted on EC2 instances to call AWS services on your behalf.
```
2. Launch a new [AWS EC2 Instance](https://us-west-2.console.aws.amazon.com/ec2/v2/home?region=us-west-2#LaunchInstanceWizard).
`GitLab Runner` will be installed on this instance and will serve as the bastion that spawns new instances. This
instance will be a dedicated host since we need it always up and running, thus it will be the standard costs apply.
Note: This doesn't have to be a powerful machine since it will not run any jobs itself, a t2.micro instance will do.
```
Amazon Machine Image (AMI): Amazon Linux AMI 2018.03.0 (HVM), SSD Volume Type - ami-0f2176987ee50226e
Instance Type: t2.micro
```
3. Configure Instance Details.
Note: Don't forget to select the IAM Role _SaasStarterKitEc2RoleForGitLabRunner_
```
Number of instances: 1
Network: default VPC
Subnet: no Preference
Auto-assign Public IP: Use subnet setting (Enable)
Placement Group: not checked/disabled
Capacity Reservation: Open
IAM Role: SaasStarterKitEc2RoleForGitLabRunner
Shutdown behavior: Stop
Enable termination project: checked/enabled
Monitoring: not checked/disabled
Tenancy: Shared
Elastic Interence: not checked/disabled
T2/T3 Unlimited: not checked/disabled
Advanced Details: none
```
4. Add Storage. Increase the volume size for the root device to 100 GiB
```
Volume Type | Device | Size (GiB) | Volume Type
Root | /dev/xvda | 100 | General Purpose SSD (gp2)
```
5. Add Tags.
```
Name: gitlab-runner
```
6. Configure Security Group. Create a new security group with the following details:
```
Name: gitlab-runner
Description: Gitlab runners for running CICD.
Rules:
Type | Protocol | Port Range | Source | Description
Custom TCP | TCP | 2376 | Anywhere | Gitlab runner for Docker Machine to communicate with Docker daemon.
SSH | TCP | 22 | My IP | SSH access for setup.
```
7. Review and Launch instance. Select an existing key pair or create a new one. This will be used to SSH into the
instance for additional configuration.
8. SSH into the newly created instance.
```bash
ssh -i ~/saas-starter-kit-uswest2-gitlabrunner.pem ec2-user@ec2-52-36-105-172.us-west-2.compute.amazonaws.com
```
Note: If you get the error `Permissions 0666 are too open`, then you will need to `chmod 400 FILENAME`
9. Install GitLab Runner from the [official GitLab repository](https://docs.gitlab.com/runner/install/linux-repository.html)
```bash
curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.rpm.sh | sudo bash
sudo yum install gitlab-runner
```
10. [Install Docker Community Edition](https://docs.docker.com/install/).
```bash
sudo yum install docker
```
11. [Install Docker Machine](https://docs.docker.com/machine/install-machine/).
```bash
base=https://github.com/docker/machine/releases/download/v0.16.0 &&
curl -L $base/docker-machine-$(uname -s)-$(uname -m) >/tmp/docker-machine &&
sudo install /tmp/docker-machine /usr/local/bin/docker-machine
```
12. [Register the runner](https://docs.gitlab.com/runner/register/index.html).
```bash
sudo gitlab-runner register
```
Notes:
* When asked for gitlab-ci tags, enter `master,dev,dev-*`
* This will limit commits to the master or dev branches from triggering the pipeline to run. This includes a
wildcard for any branch named with the prefix `dev-`.
* When asked the executor type, enter `docker+machine`
* When asked for the default Docker image, enter `golang:alpine3.9`
13. [Configuring the GitLab Runner](https://docs.gitlab.com/runner/configuration/runner_autoscale_aws/#configuring-the-gitlab-runner)
```bash
sudo vim /etc/gitlab-runner/config.toml
```
Update the `[runners.docker]` configuration section in `config.toml` to match the example below replacing the
obvious placeholder `XXXXX` with the relevant value.
```yaml
[runners.docker]
tls_verify = false
image = "golang:alpine3.9"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = true
volumes = ["/cache"]
shm_size = 0
[runners.cache]
Type = "s3"
Shared = true
[runners.cache.s3]
ServerAddress = "s3.us-west-2.amazonaws.com"
BucketName = "XXXXX"
BucketLocation = "us-west-2"
[runners.machine]
IdleCount = 0
IdleTime = 1800
MachineDriver = "amazonec2"
MachineName = "gitlab-runner-machine-%s"
MachineOptions = [
"amazonec2-iam-instance-profile=SaasStarterKitEc2RoleForGitLabRunner",
"amazonec2-region=us-west-2",
"amazonec2-vpc-id=XXXXX",
"amazonec2-subnet-id=XXXXX",
"amazonec2-zone=d",
"amazonec2-use-private-address=true",
"amazonec2-tags=runner-manager-name,gitlab-aws-autoscaler,gitlab,true,gitlab-runner-autoscale,true",
"amazonec2-security-group=gitlab-runner",
"amazonec2-instance-type=t2.large"
]
```
You will need use the same VPC subnet and availability zone as the instance launched in step 2. We are using AWS
region `us-west-2`. The _ServerAddress_ for S3 will need to be updated if the region is changed. For `us-east-1` the
_ServerAddress_ is `s3.amazonaws.com`. Under MachineOptions you can add anything that the [AWS Docker Machine](https://docs.docker.com/machine/drivers/aws/#options)
driver supports.
Below are some example values for the placeholders to ensure for format of your values are correct.
```yaml
BucketName = saas-starter-kit-usw
amazonec2-vpc-id=vpc-5f43f027
amazonec2-subnet-id=subnet-693d3110
amazonec2-zone=a
```
Once complete, restart the runner.
```bash
sudo gitlab-runner restart
```
## What's Next
We are in the process of writing more documentation about this code. We welcome you to make enhancements to this documentation or just send us your feedback and suggestions ; )

2
cmd/web-api/handlers/check.go
View File

@ -47,5 +47,5 @@ func (c *Check) Ping(ctx context.Context, w http.ResponseWriter, r *http.Request
status := "pong"
return web.RespondJson(ctx, w, status, http.StatusOK)
return web.RespondText(ctx, w, status, http.StatusOK)
}

8
cmd/web-app/handlers/check.go
View File

@ -40,3 +40,11 @@ func (c *Check) Health(ctx context.Context, w http.ResponseWriter, r *http.Reque
return c.Renderer.Render(ctx, w, r, baseLayoutTmpl, "health.tmpl", web.MIMETextHTMLCharsetUTF8, http.StatusOK, data)
}
// Ping validates the service is ready to accept requests.
func (c *Check) Ping(ctx context.Context, w http.ResponseWriter, r *http.Request, params map[string]string) error {
status := "pong"
return web.RespondText(ctx, w, status, http.StatusOK)
}

5
cmd/web-app/handlers/routes.go
View File

@ -6,15 +6,15 @@ import (
"os"
"geeks-accelerator/oss/saas-starter-kit/internal/mid"
"geeks-accelerator/oss/saas-starter-kit/internal/platform/auth"
"geeks-accelerator/oss/saas-starter-kit/internal/platform/web"
"github.com/jmoiron/sqlx"
"gopkg.in/DataDog/dd-trace-go.v1/contrib/go-redis/redis"
)
const baseLayoutTmpl = "base.tmpl"
// API returns a handler for a set of routes.
func APP(shutdown chan os.Signal, log *log.Logger, staticDir, templateDir string, masterDB *sqlx.DB, authenticator *auth.Authenticator, renderer web.Renderer, globalMids ...web.Middleware) http.Handler {
func APP(shutdown chan os.Signal, log *log.Logger, staticDir, templateDir string, masterDB *sqlx.DB, redis *redis.Client, renderer web.Renderer, globalMids ...web.Middleware) http.Handler {
// Define base middlewares applied to all requests.
middlewares := []web.Middleware{
@ -32,6 +32,7 @@ func APP(shutdown chan os.Signal, log *log.Logger, staticDir, templateDir string
// Register health check endpoint. This route is not authenticated.
check := Check{
MasterDB: masterDB,
Redis: redis,
Renderer: renderer,
}
app.Handle("GET", "/v1/health", check.Health)

292
cmd/web-app/main.go
View File

@ -2,10 +2,13 @@ package main
import (
"context"
"crypto/tls"
"encoding/json"
"expvar"
"fmt"
"geeks-accelerator/oss/saas-starter-kit/internal/mid"
"golang.org/x/crypto/acme"
"golang.org/x/crypto/acme/autocert"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer"
"html/template"
"log"
@ -67,15 +70,18 @@ func main() {
Host string `default:"" envconfig:"HOST"`
ReadTimeout time.Duration `default:"5s" envconfig:"READ_TIMEOUT"`
WriteTimeout time.Duration `default:"5s" envconfig:"WRITE_TIMEOUT"`
DisableHTTP2 bool `default:"false" envconfig:"DISABLE_HTTP2"`
}
App struct {
Name string `default:"web-app" envconfig:"NAME"`
BaseUrl string `default:"" envconfig:"BASE_URL"`
TemplateDir string `default:"./templates" envconfig:"TEMPLATE_DIR"`
StaticDir string `default:"./static" envconfig:"STATIC_DIR"`
Service struct {
Name string `default:"web-app" envconfig:"NAME"`
Project string `default:"" envconfig:"PROJECT"`
BaseUrl string `default:"" envconfig:"BASE_URL" example:"http://eproc.tech"`
HostNames []string `envconfig:"HOST_NAMES" example:"www.eproc.tech"`
EnableHTTPS bool `default:"false" envconfig:"ENABLE_HTTPS"`
TemplateDir string `default:"./templates" envconfig:"TEMPLATE_DIR"`
StaticDir string `default:"./static" envconfig:"STATIC_DIR"`
StaticS3 struct {
S3Enabled bool `envconfig:"ENABLED"`
S3Bucket string `envconfig:"S3_BUCKET"`
S3KeyPrefix string `default:"public/web_app/static" envconfig:"KEY_PREFIX"`
CloudFrontEnabled bool `envconfig:"CLOUDFRONT_ENABLED"`
ImgResizeEnabled bool `envconfig:"IMG_RESIZE_ENABLED"`
@ -104,9 +110,12 @@ func main() {
AnalyticsRate float64 `default:"0.10" envconfig:"ANALYTICS_RATE"`
}
Aws struct {
AccessKeyID string `envconfig:"AWS_ACCESS_KEY_ID" required:"true"` // WEB_API_AWS_AWS_ACCESS_KEY_ID or AWS_ACCESS_KEY_ID
SecretAccessKey string `envconfig:"AWS_SECRET_ACCESS_KEY" required:"true" json:"-"` // don't print
Region string `default:"us-east-1" envconfig:"AWS_REGION"`
AccessKeyID string `envconfig:"AWS_ACCESS_KEY_ID"` // WEB_API_AWS_AWS_ACCESS_KEY_ID or AWS_ACCESS_KEY_ID
SecretAccessKey string `envconfig:"AWS_SECRET_ACCESS_KEY" json:"-"` // don't print
Region string `default:"us-east-1" envconfig:"AWS_REGION"`
S3BucketPrivate string `envconfig:"S3_BUCKET_PRIVATE"`
S3BucketPublic string `envconfig:"S3_BUCKET_PUBLIC"`
SecretsManagerConfigPrefix string `default:"" envconfig:"SECRETS_MANAGER_CONFIG_PREFIX"`
// Get an AWS session from an implicit source if no explicit
// configuration is provided. This is useful for taking advantage of
@ -114,8 +123,7 @@ func main() {
UseRole bool `envconfig:"AWS_USE_ROLE"`
}
Auth struct {
UseAwsSecretManager bool `default:false envconfig:"USE_AWS_SECRET_MANAGER"`
AwsSecretID string `default:"auth-secret-key" envconfig:"AWS_SECRET_ID"`
UseAwsSecretManager bool `default:"false" envconfig:"USE_AWS_SECRET_MANAGER"`
KeyExpiration time.Duration `default:"3600s" envconfig:"KEY_EXPIRATION"`
}
BuildInfo struct {
@ -130,7 +138,6 @@ func main() {
CiPipelineId string `envconfig:"CI_COMMIT_PIPELINE_ID"`
CiPipelineUrl string `envconfig:"CI_COMMIT_PIPELINE_URL"`
}
CMD string `envconfig:"CMD"`
}
// For additional details refer to https://github.com/kelseyhightower/envconfig
@ -154,8 +161,20 @@ func main() {
cfg.Aws.SecretAccessKey = ""
}
// Set the default AWS Secrets Manager prefix used for name to store config files that will be persisted across
// deployments and distributed to each instance of the service running.
if cfg.Aws.SecretsManagerConfigPrefix == "" {
var pts []string
if cfg.Service.Project != "" {
pts = append(pts, cfg.Service.Project)
}
pts = append(pts, cfg.Env, cfg.Service.Name)
cfg.Aws.SecretsManagerConfigPrefix = filepath.Join(pts...)
}
// If base URL is empty, set the default value from the HTTP Host
if cfg.App.BaseUrl == "" {
if cfg.Service.BaseUrl == "" {
baseUrl := cfg.HTTP.Host
if !strings.HasPrefix(baseUrl, "http") {
if strings.HasPrefix(baseUrl, "0.0.0.0:") {
@ -167,15 +186,37 @@ func main() {
}
baseUrl = "http://" + baseUrl
}
cfg.App.BaseUrl = baseUrl
cfg.Service.BaseUrl = baseUrl
}
// When HTTPS is not specifically enabled, but an HTTP host is set, enable HTTPS.
if !cfg.Service.EnableHTTPS && cfg.HTTPS.Host != "" {
cfg.Service.EnableHTTPS = true
}
// Determine the primary host by parsing host from the base app URL.
baseSiteUrl, err := url.Parse(cfg.Service.BaseUrl)
if err != nil {
log.Fatalf("main : Parse service base URL : %s : %+v", cfg.Service.BaseUrl, err)
}
// Drop any ports from the base app URL.
var primaryServiceHost string
if strings.Contains(baseSiteUrl.Host, ":") {
primaryServiceHost, _, err = net.SplitHostPort(baseSiteUrl.Host)
if err != nil {
log.Fatalf("main : SplitHostPort : %s : %+v", baseSiteUrl.Host, err)
}
} else {
primaryServiceHost = baseSiteUrl.Host
}
// =========================================================================
// Log App Info
// Log Service Info
// Print the build version for our logs. Also expose it under /debug/vars.
expvar.NewString("build").Set(build)
log.Printf("main : Started : Application Initializing version %q", build)
log.Printf("main : Started : Service Initializing version %q", build)
defer log.Println("main : Completed")
// Print the config for our logs. It's important to any credentials in the config
@ -197,11 +238,20 @@ func main() {
// configuration is provided. This is useful for taking advantage of
// EC2/ECS instance roles.
awsSession = session.Must(session.NewSession())
} else {
log.Printf("main : AWS : Using role.\n")
} else if cfg.Aws.AccessKeyID != "" {
creds := credentials.NewStaticCredentials(cfg.Aws.AccessKeyID, cfg.Aws.SecretAccessKey, "")
awsSession = session.New(&aws.Config{Region: aws.String(cfg.Aws.Region), Credentials: creds})
log.Printf("main : AWS : Using static credentials\n")
}
// Wrap the AWS session to enable tracing.
if awsSession != nil {
awsSession = awstrace.WrapSession(awsSession)
}
awsSession = awstrace.WrapSession(awsSession)
// =========================================================================
// Start Redis
@ -274,49 +324,35 @@ func main() {
defer masterDb.Close()
// =========================================================================
// Deploy
switch cfg.CMD {
case "sync-static":
// sync static files to S3
if cfg.App.StaticS3.S3Enabled || cfg.App.StaticS3.CloudFrontEnabled {
err = devops.SyncS3StaticFiles(awsSession, cfg.App.StaticS3.S3Bucket, cfg.App.StaticS3.S3KeyPrefix, cfg.App.StaticDir)
if err != nil {
log.Fatalf("main : deploy : %+v", err)
}
}
return
// Load middlewares that need to be configured specific for the service.
var serviceMiddlewares []web.Middleware
// Init redirect middleware to ensure all requests go to the primary domain contained in the base URL.
if primaryServiceHost != "127.0.0.0" && primaryServiceHost != "localhost" {
redirect := mid.DomainNameRedirect(mid.DomainNameRedirectConfig{
RedirectConfig: mid.RedirectConfig{
Code: http.StatusMovedPermanently,
Skipper: func(ctx context.Context, w http.ResponseWriter, r *http.Request, params map[string]string) bool {
if r.URL.Path == "/ping" {
return true
}
return false
},
},
DomainName: primaryServiceHost,
HTTPSEnabled: cfg.Service.EnableHTTPS,
})
serviceMiddlewares = append(serviceMiddlewares, redirect)
}
// =========================================================================
// Init redirect middleware to ensure all requests go to the primary domain.
baseSiteUrl, err := url.Parse(cfg.App.BaseUrl)
if err != nil {
log.Fatalf("main : Parse App Base URL : %s : %+v", cfg.App.BaseUrl, err)
}
var primaryDomain string
if strings.Contains(baseSiteUrl.Host, ":") {
primaryDomain, _, err = net.SplitHostPort(baseSiteUrl.Host)
if err != nil {
log.Fatalf("main : SplitHostPort : %s : %+v", baseSiteUrl.Host, err)
}
} else {
primaryDomain = baseSiteUrl.Host
}
redirect := mid.DomainNameRedirect(mid.DomainNameRedirectConfig{
DomainName: primaryDomain,
HTTPSEnabled: (cfg.HTTPS.Host != ""),
})
// =========================================================================
// URL Formatter
// s3UrlFormatter is a help function used by to convert an s3 key to
// a publicly available image URL.
var staticS3UrlFormatter func(string) string
if cfg.App.StaticS3.S3Enabled || cfg.App.StaticS3.CloudFrontEnabled || cfg.App.StaticS3.ImgResizeEnabled {
s3UrlFormatter, err := devops.S3UrlFormatter(awsSession, cfg.App.StaticS3.S3Bucket, cfg.App.StaticS3.S3KeyPrefix, cfg.App.StaticS3.CloudFrontEnabled)
if cfg.Service.StaticS3.S3Enabled || cfg.Service.StaticS3.CloudFrontEnabled || cfg.Service.StaticS3.ImgResizeEnabled {
s3UrlFormatter, err := devops.S3UrlFormatter(awsSession, cfg.Aws.S3BucketPublic, cfg.Service.StaticS3.S3KeyPrefix, cfg.Service.StaticS3.CloudFrontEnabled)
if err != nil {
log.Fatalf("main : S3UrlFormatter failed : %+v", err)
}
@ -325,10 +361,20 @@ func main() {
// When the path starts with a forward slash its referencing a local file,
// make sure the static file prefix is included
if strings.HasPrefix(p, "/") {
p = filepath.Join(cfg.App.StaticS3.S3KeyPrefix, p)
p = filepath.Join(cfg.Service.StaticS3.S3KeyPrefix, p)
}
return s3UrlFormatter(p)
}
} else {
baseUrl, err := url.Parse(cfg.Service.BaseUrl)
if err != nil {
log.Fatalf("main : url Parse(%s) : %+v", cfg.Service.BaseUrl, err)
}
staticS3UrlFormatter = func(p string) string {
baseUrl.Path = p
return baseUrl.String()
}
}
// staticUrlFormatter is a help function used by template functions defined below.
@ -336,12 +382,12 @@ func main() {
// templates should be updated to use a fully qualified URL for either the public file on S3
// on from the cloudfront distribution.
var staticUrlFormatter func(string) string
if cfg.App.StaticS3.S3Enabled || cfg.App.StaticS3.CloudFrontEnabled {
if cfg.Service.StaticS3.S3Enabled || cfg.Service.StaticS3.CloudFrontEnabled {
staticUrlFormatter = staticS3UrlFormatter
} else {
baseUrl, err := url.Parse(cfg.App.BaseUrl)
baseUrl, err := url.Parse(cfg.Service.BaseUrl)
if err != nil {
log.Fatalf("main : url Parse(%s) : %+v", cfg.App.BaseUrl, err)
log.Fatalf("main : url Parse(%s) : %+v", cfg.Service.BaseUrl, err)
}
staticUrlFormatter = func(p string) string {
@ -410,7 +456,7 @@ func main() {
"SiteAssetUrl": func(p string) string {
var u string
if staticUrlFormatter != nil {
u = staticUrlFormatter(filepath.Join(cfg.App.Name, p))
u = staticUrlFormatter(filepath.Join(cfg.Service.Name, p))
} else {
if !strings.HasPrefix(p, "/") {
p = "/" + p
@ -425,7 +471,7 @@ func main() {
"SiteS3Url": func(p string) string {
var u string
if staticUrlFormatter != nil {
u = staticUrlFormatter(filepath.Join(cfg.App.Name, p))
u = staticUrlFormatter(filepath.Join(cfg.Service.Name, p))
} else {
u = p
}
@ -444,13 +490,13 @@ func main() {
// Image Formatter - additional functions exposed to templates for resizing images
// to support response web applications.
imgResizeS3KeyPrefix := filepath.Join(cfg.App.StaticS3.S3KeyPrefix, "images/responsive")
imgResizeS3KeyPrefix := filepath.Join(cfg.Service.StaticS3.S3KeyPrefix, "images/responsive")
imgSrcAttr := func(ctx context.Context, p string, sizes []int, includeOrig bool) template.HTMLAttr {
u := staticUrlFormatter(p)
var srcAttr string
if cfg.App.StaticS3.ImgResizeEnabled {
srcAttr, _ = img_resize.S3ImgSrc(ctx, redisClient, staticS3UrlFormatter, awsSession, cfg.App.StaticS3.S3Bucket, imgResizeS3KeyPrefix, u, sizes, includeOrig)
if cfg.Service.StaticS3.ImgResizeEnabled {
srcAttr, _ = img_resize.S3ImgSrc(ctx, redisClient, staticS3UrlFormatter, awsSession, cfg.Aws.S3BucketPublic, imgResizeS3KeyPrefix, u, sizes, includeOrig)
} else {
srcAttr = fmt.Sprintf("src=\"%s\"", u)
}
@ -480,8 +526,8 @@ func main() {
}
tmplFuncs["S3ImgUrl"] = func(ctx context.Context, p string, size int) string {
imgUrl := staticUrlFormatter(p)
if cfg.App.StaticS3.ImgResizeEnabled {
imgUrl, _ = img_resize.S3ImgUrl(ctx, redisClient, staticS3UrlFormatter, awsSession, cfg.App.StaticS3.S3Bucket, imgResizeS3KeyPrefix, imgUrl, size)
if cfg.Service.StaticS3.ImgResizeEnabled {
imgUrl, _ = img_resize.S3ImgUrl(ctx, redisClient, staticS3UrlFormatter, awsSession, cfg.Aws.S3BucketPublic, imgResizeS3KeyPrefix, imgUrl, size)
}
return imgUrl
}
@ -519,7 +565,7 @@ func main() {
enableHotReload := cfg.Env == "dev"
// Template Renderer used to generate HTML response for web experience.
renderer, err := template_renderer.NewTemplateRenderer(cfg.App.TemplateDir, enableHotReload, gvd, t, eh)
renderer, err := template_renderer.NewTemplateRenderer(cfg.Service.TemplateDir, enableHotReload, gvd, t, eh)
if err != nil {
log.Fatalf("main : Marshalling Config to JSON : %+v", err)
}
@ -538,13 +584,20 @@ func main() {
//
// /debug/vars - Added to the default mux by the expvars package.
// /debug/pprof - Added to the default mux by the net/http/pprof package.
if cfg.App.DebugHost != "" {
if cfg.Service.DebugHost != "" {
go func() {
log.Printf("main : Debug Listening %s", cfg.App.DebugHost)
log.Printf("main : Debug Listener closed : %v", http.ListenAndServe(cfg.App.DebugHost, http.DefaultServeMux))
log.Printf("main : Debug Listening %s", cfg.Service.DebugHost)
log.Printf("main : Debug Listener closed : %v", http.ListenAndServe(cfg.Service.DebugHost, http.DefaultServeMux))
}()
}
// =========================================================================
// ECS Task registration for services that don't use an AWS Elastic Load Balancer.
err = devops.EcsServiceTaskInit(log, awsSession)
if err != nil {
log.Fatalf("main : Ecs Service Task init : %+v", err)
}
// =========================================================================
// Start APP Service
@ -553,23 +606,81 @@ func main() {
shutdown := make(chan os.Signal, 1)
signal.Notify(shutdown, os.Interrupt, syscall.SIGTERM)
app := http.Server{
Addr: cfg.HTTP.Host,
Handler: handlers.APP(shutdown, log, cfg.App.StaticDir, cfg.App.TemplateDir, masterDb, nil, renderer, redirect),
ReadTimeout: cfg.HTTP.ReadTimeout,
WriteTimeout: cfg.HTTP.WriteTimeout,
MaxHeaderBytes: 1 << 20,
}
// Make a channel to listen for errors coming from the listener. Use a
// buffered channel so the goroutine can exit if we don't collect this error.
serverErrors := make(chan error, 1)
// Start the service listening for requests.
go func() {
log.Printf("main : APP Listening %s", cfg.HTTP.Host)
serverErrors <- app.ListenAndServe()
}()
// Make an list of HTTP servers for both HTTP and HTTPS requests.
var httpServers []http.Server
// Start the HTTP service listening for requests.
if cfg.HTTP.Host != "" {
api := http.Server{
Addr: cfg.HTTP.Host,
Handler: handlers.APP(shutdown, log, cfg.Service.StaticDir, cfg.Service.TemplateDir, masterDb, redisClient, renderer, serviceMiddlewares...),
ReadTimeout: cfg.HTTP.ReadTimeout,
WriteTimeout: cfg.HTTP.WriteTimeout,
MaxHeaderBytes: 1 << 20,
}
httpServers = append(httpServers, api)
go func() {
log.Printf("main : APP Listening %s", cfg.HTTP.Host)
serverErrors <- api.ListenAndServe()
}()
}
// Start the HTTPS service listening for requests with an SSL Cert auto generated with Let's Encrypt.
if cfg.HTTPS.Host != "" {
api := http.Server{
Addr: cfg.HTTPS.Host,
Handler: handlers.APP(shutdown, log, cfg.Service.StaticDir, cfg.Service.TemplateDir, masterDb, redisClient, renderer, serviceMiddlewares...),
ReadTimeout: cfg.HTTPS.ReadTimeout,
WriteTimeout: cfg.HTTPS.WriteTimeout,
MaxHeaderBytes: 1 << 20,
}
// Generate a unique list of hostnames.
var hosts []string
if primaryServiceHost != "" {
hosts = append(hosts, primaryServiceHost)
}
for _, h := range cfg.Service.HostNames {
h = strings.TrimSpace(h)
if h != "" && h != primaryServiceHost {
hosts = append(hosts, h)
}
}
// Enable autocert to store certs via Secret Manager.
secretPrefix := filepath.Join(cfg.Aws.SecretsManagerConfigPrefix, "autocert")
// Local file cache to reduce requests hitting Secret Manager.
localCache := autocert.DirCache(os.TempDir())
cache, err := devops.NewSecretManagerAutocertCache(log, awsSession, secretPrefix, localCache)
if err != nil {
log.Fatalf("main : HTTPS : %+v", err)
}
m := &autocert.Manager{
Prompt: autocert.AcceptTOS,
HostPolicy: autocert.HostWhitelist(hosts...),
Cache: cache,
}
api.TLSConfig = &tls.Config{GetCertificate: m.GetCertificate}
api.TLSConfig.NextProtos = append(api.TLSConfig.NextProtos, acme.ALPNProto)
if !cfg.HTTPS.DisableHTTP2 {
api.TLSConfig.NextProtos = append(api.TLSConfig.NextProtos, "h2")
}
httpServers = append(httpServers, api)
go func() {
log.Printf("main : APP Listening %s with SSL cert for hosts %s", cfg.HTTPS.Host, strings.Join(hosts, ", "))
serverErrors <- api.ListenAndServeTLS("", "")
}()
}
// =========================================================================
// Shutdown
@ -583,14 +694,19 @@ func main() {
log.Printf("main : %v : Start shutdown..", sig)
// Create context for Shutdown call.
ctx, cancel := context.WithTimeout(context.Background(), cfg.App.ShutdownTimeout)
ctx, cancel := context.WithTimeout(context.Background(), cfg.Service.ShutdownTimeout)
defer cancel()
// Asking listener to shutdown and load shed.
err := app.Shutdown(ctx)
if err != nil {
log.Printf("main : Graceful shutdown did not complete in %v : %v", cfg.App.ShutdownTimeout, err)
err = app.Close()
// Handle closing connections for both possible HTTP servers.
for _, api := range httpServers {
// Asking listener to shutdown and load shed.
err := api.Shutdown(ctx)
if err != nil {
log.Printf("main : Graceful shutdown did not complete in %v : %v", cfg.Service.ShutdownTimeout, err)
err = api.Close()
}
}
// Log the status of this shutdown.

9
internal/platform/web/response.go
View File

@ -118,6 +118,15 @@ func RespondErrorStatus(ctx context.Context, w http.ResponseWriter, er error, st
return nil
}
// RespondText sends text back to the client as plain text with the specified HTTP status code.
func RespondText(ctx context.Context, w http.ResponseWriter, text string, statusCode int) error {
if err := Respond(ctx, w, []byte(text), statusCode, MIMETextPlainCharsetUTF8); err != nil {
return err
}
return nil
}
// Respond writes the data to the client with the specified HTTP status code and
// content type.
func Respond(ctx context.Context, w http.ResponseWriter, data []byte, statusCode int, contentType string) error {

10
sample.env_docker_compose
View File

@ -1,5 +1,5 @@
AWS_ACCESS_KEY_ID=XXXX
AWS_SECRET_ACCESS_KEY=XXXX
AWS_REGION=us-east-1
AWS_USE_ROLE=false
DD_API_KEY=XXXX
#AWS_ACCESS_KEY_ID=
#AWS_SECRET_ACCESS_KEY=
#AWS_REGION=us-west-2
#AWS_USE_ROLE=false
#DD_API_KEY=