From 5291ddd7266d059b0940bca2718a6631caeaef4b Mon Sep 17 00:00:00 2001 From: Lee Brown Date: Wed, 21 Aug 2019 20:33:54 -0800 Subject: [PATCH] lower geonames batch size --- README.md | 67 ++++++++++++++++++++++++++++++++--- internal/schema/migrations.go | 2 +- 2 files changed, 63 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 390859b..5900d72 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ # SaaS Startup Kit +[![Build Status](https://gitlab.com/geeks-accelerator/oss/saas-starter-kit/badges/master/pipeline.svg)](https://gitlab.com/geeks-accelerator/oss/devops/pipelines) +[![Go Report Card](https://goreportcard.com/badge/gitlab.com/geeks-accelerator/oss/saas-starter-kit?style=flat-square)](https://goreportcard.com/report/gitlab.com/geeks-accelerator/oss/devops) -Copyright 2019, Geeks Accelerator -twins@geeksaccelerator.com The [SaaS Startup Kit](https://saasstartupkit.com/) is a set of libraries in Go and boilerplate Golang code for building scalable software-as-a-service (SaaS) applications. The goal of this project is to provide a proven starting point for new @@ -26,6 +26,38 @@ https://docs.google.com/presentation/d/1WGYqMZ-YUOaNxlZBfU4srpN8i86MU0ppWWSBb3pk [![Google Slides of Screen Captures for SaaS Startup Kit web app](resources/images/saas-webapp-screencapture-01.jpg)](https://saasstartupkit.com/) + + + +- [Motivation](#motivation) +- [Description](#description) + * [Example project](#example-project) +- [Local Installation](#local-installation) + * [Getting the project](#getting-the-project) + * [Go Modules](#go-modules) + * [Installing Docker](#installing-docker) +- [Getting started](#getting-started) + * [Running the project](#running-the-project) + * [How we run the project](#how-we-run-the-project) + * [Stopping the project](#stopping-the-project) + * [Re-starting a specific Go service for development](#re-starting-a-specific-go-service-for-development) + * [Forking your own copy](#forking-your-own-copy) + * [Optional. Set AWS and Datadog Configs](#optional-set-aws-and-datadog-configs) +- [Web API](#web-api) + * [API Documentation](#api-documentation) +- [Web App](#web-app) +- [Schema](#schema) + * [Accessing Postgres](#accessing-postgres) +- [Deployment](#deployment) +- [Development Notes](#development-notes) +- [What's Next](#whats-next) +- [Join us on Gopher Slack](#join-us-on-gopher-slack) +- [License](#license) + + + + + ## Motivation When getting started building SaaS, we believe that is important for both the frontend web experience and the backend @@ -199,8 +231,8 @@ Docker is a critical component and required to run this project. https://docs.docker.com/install/ - -## Running The Project + +## Getting started There is a `docker-compose` file that knows how to build and run all the services. Each service has its own a `dockerfile`. @@ -490,6 +522,7 @@ of user management. For more details on this service, read [web-app readme](https://gitlab.com/geeks-accelerator/oss/saas-starter-kit/blob/master/cmd/web-app/README.md) + ## Schema [cmd/schema](https://gitlab.com/geeks-accelerator/oss/saas-starter-kit/tree/master/cmd/schema) @@ -622,13 +655,37 @@ sqlQueryStr = db.Rebind(sqlQueryStr) For additional details refer to [bindvars](https://jmoiron.github.io/sqlx/#bindvars) + ## What's Next We are in the process of writing more documentation about this code. We welcome you to make enhancements to this documentation or just send us your feedback and suggestions :wink: + +## Contributions + +We :heart: contributions. + +Have you had a good experience with SaaS Startup Kit? Why not share some love and contribute code? + +Thank you to all those that have contributed to this project and are using it in their projects. You can find a +CONTRIBUTORS file where we keep a list of contributors to the project. If you contribute a PR please consider adding +your name there. + + + ## Join us on Gopher Slack If you are having problems installing, troubles getting the project running or would like to contribute, join the -channel #saas-starter-kit on [Gopher Slack](http://invite.slack.golangbridge.org/) \ No newline at end of file +channel #saas-starter-kit on [Gopher Slack](http://invite.slack.golangbridge.org/) + + + +## License + +Please read the [LICENSE](./LICENSE) file here. + + +Copyright 2019, Geeks Accelerator +twins@geeksaccelerator.com diff --git a/internal/schema/migrations.go b/internal/schema/migrations.go index 6d34397..5cea8a7 100644 --- a/internal/schema/migrations.go +++ b/internal/schema/migrations.go @@ -294,7 +294,7 @@ func migrationList(ctx context.Context, db *sqlx.DB, log *log.Logger, isUnittest } //fmt.Println("Geoname records: ", len(v)) // Max argument values of Postgres is about 54460. So the batch size for bulk insert is selected 4500*12 (ncol) - batch := 4500 + batch := 1000 n := len(v) / batch //fmt.Println("Number of batch: ", n)