go/ko-build
1
0
Fork 0
mirror of https://github.com/ko-build/ko.git synced 2025-11-06 09:19:12 +02:00
Code Issues Releases Activity

Initial commit

Browse Source
This commit is contained in:
Jason Hall
2019-03-14 14:23:47 -04:00
commit 6354665a42
1848 changed files with 881447 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

420
cmd/README.md Normal file
View File

@@ -0,0 +1,420 @@
# `ko`
`ko` is a CLI to support rapid development of Go containers
with Kubernetes and minimal configuration.
## Installation
`ko` can be installed via:
```shell
go get github.com/google/go-containerregistry/cmd/ko
```
To update your installation:
```shell
go get -u github.com/google/go-containerregistry/cmd/ko
```
## The `ko` Model
`ko` is built around a very simple extension to Go's model for expressing
dependencies using [import paths](https://golang.org/doc/code.html#ImportPaths).
In Go, dependencies are expressed via blocks like:
```go
import (
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"
)
```
Similarly (as you can see above), Go binaries can be referenced via import
paths like `github.com/google/go-containerregistry/cmd/ko`.
**One of the goals of `ko` is to make containers invisible infrastructure.**
Simply replace image references in your Kubernetes yaml with the import path for
your Go binary, and `ko` will handle containerizing and publishing that
container image as needed.
For example, you might use the following in a Kubernetes `Deployment` resource:
```yaml
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: hello-world
spec:
selector:
matchLabels:
foo: bar
replicas: 1
template:
metadata:
labels:
foo: bar
spec:
containers:
- name: hello-world
# This is the import path for the Go binary to build and run.
image: github.com/mattmoor/examples/http/cmd/helloworld
ports:
- containerPort: 8080
```
### Determining supported import paths
Similar to other tooling in the Go ecosystem, `ko` expects to execute in the
context of your `$GOPATH`. This is used to determine what package(s) `ko`
is expected to build.
Suppose `GOPATH` is `~/gopath` and the current directory is
`~/gopath/src/github.com/mattmoor/examples`. `ko` will deduce the base import
path to be `github.com/mattmoor/examples`, and any references to subpackages
of this will be built, containerized and published.
For example, any of the following would be matched:
* `github.com/mattmoor/examples`
* `github.com/mattmoor/examples/cmd/foo`
* `github.com/mattmoor/examples/bar`
### Results
Employing this convention enables `ko` to have effectively zero configuration
and enable very fast development iteration. For
[warm-image](https://github.com/mattmoor/warm-image), `ko` is able to
build, containerize, and redeploy a non-trivial Kubernetes controller app in
seconds (dominated by two `go build`s).
```shell
$ ko apply -f config/
2018/07/19 14:56:41 Using base gcr.io/distroless/base:latest for github.com/mattmoor/warm-image/cmd/sleeper
2018/07/19 14:56:42 Publishing us.gcr.io/my-project/sleeper-ebdb8b8b13d4bbe1d3592de055016d37:latest
2018/07/19 14:56:43 mounted blob: sha256:57752e7f9593cbfb7101af994b136a369ecc8174332866622db32a264f3fbefd
2018/07/19 14:56:43 mounted blob: sha256:59df9d5b488aea2753ab7774ae41a9a3e96903f87ac699f3505960e744f36f7d
2018/07/19 14:56:43 mounted blob: sha256:739b3deec2edb17c512f507894c55c2681f9724191d820cdc01f668330724ca7
2018/07/19 14:56:44 us.gcr.io/my-project/sleeper-ebdb8b8b13d4bbe1d3592de055016d37:latest: digest: sha256:6c7b96a294cad3ce613aac23c8aca5f9dd12a894354ab276c157fb5c1c2e3326 size: 592
2018/07/19 14:56:44 Published us.gcr.io/my-project/sleeper-ebdb8b8b13d4bbe1d3592de055016d37@sha256:6c7b96a294cad3ce613aac23c8aca5f9dd12a894354ab276c157fb5c1c2e3326
2018/07/19 14:56:45 Using base gcr.io/distroless/base:latest for github.com/mattmoor/warm-image/cmd/controller
2018/07/19 14:56:46 Publishing us.gcr.io/my-project/controller-9e91872fd7c48124dbe6ea83944b87e9:latest
2018/07/19 14:56:46 mounted blob: sha256:007782ba6738188a59bf21b4d8e974f218615ee948c6357535d07e7248b2a560
2018/07/19 14:56:46 mounted blob: sha256:57752e7f9593cbfb7101af994b136a369ecc8174332866622db32a264f3fbefd
2018/07/19 14:56:46 mounted blob: sha256:7fec050f965d7fba3de4bd19739746dce5a5125331b7845bf02185ff5d4cc374
2018/07/19 14:56:47 us.gcr.io/my-project/controller-9e91872fd7c48124dbe6ea83944b87e9:latest: digest: sha256:5a81029bb0cfd519c321aeeea2bc1b7dc6488b6c72003d3613442b4d5e4ed14d size: 593
2018/07/19 14:56:47 Published us.gcr.io/my-project/controller-9e91872fd7c48124dbe6ea83944b87e9@sha256:5a81029bb0cfd519c321aeeea2bc1b7dc6488b6c72003d3613442b4d5e4ed14d
namespace/warmimage-system configured
clusterrolebinding.rbac.authorization.k8s.io/warmimage-controller-admin configured
deployment.apps/warmimage-controller unchanged
serviceaccount/warmimage-controller unchanged
customresourcedefinition.apiextensions.k8s.io/warmimages.mattmoor.io configured
```
## Usage
`ko` has four commands, most of which build and publish images as part of
their execution. By default, `ko` publishes images to a Docker Registry
specified via `KO_DOCKER_REPO`.
However, these same commands can be directed to operate locally as well via
the `--local` or `-L` command (or setting `KO_DOCKER_REPO=ko.local`). See
the [`minikube` section](./README.md#with-minikube) for more detail.
### `ko publish`
`ko publish` simply builds and publishes images for each import path passed as
an argument. It prints the images' published digests after each image is published.
```shell
$ ko publish github.com/mattmoor/warm-image/cmd/sleeper
2018/07/19 14:57:34 Using base gcr.io/distroless/base:latest for github.com/mattmoor/warm-image/cmd/sleeper
2018/07/19 14:57:35 Publishing us.gcr.io/my-project/sleeper-ebdb8b8b13d4bbe1d3592de055016d37:latest
2018/07/19 14:57:35 mounted blob: sha256:739b3deec2edb17c512f507894c55c2681f9724191d820cdc01f668330724ca7
2018/07/19 14:57:35 mounted blob: sha256:57752e7f9593cbfb7101af994b136a369ecc8174332866622db32a264f3fbefd
2018/07/19 14:57:35 mounted blob: sha256:59df9d5b488aea2753ab7774ae41a9a3e96903f87ac699f3505960e744f36f7d
2018/07/19 14:57:36 us.gcr.io/my-project/sleeper-ebdb8b8b13d4bbe1d3592de055016d37:latest: digest: sha256:6c7b96a294cad3ce613aac23c8aca5f9dd12a894354ab276c157fb5c1c2e3326 size: 592
2018/07/19 14:57:36 Published us.gcr.io/my-project/sleeper-ebdb8b8b13d4bbe1d3592de055016d37@sha256:6c7b96a294cad3ce613aac23c8aca5f9dd12a894354ab276c157fb5c1c2e3326
```
`ko publish` also supports relative import paths, when in the context of a repo on `GOPATH`.
```shell
$ ko publish ./cmd/sleeper
2018/07/19 14:58:16 Using base gcr.io/distroless/base:latest for github.com/mattmoor/warm-image/cmd/sleeper
2018/07/19 14:58:16 Publishing us.gcr.io/my-project/sleeper-ebdb8b8b13d4bbe1d3592de055016d37:latest
2018/07/19 14:58:17 mounted blob: sha256:59df9d5b488aea2753ab7774ae41a9a3e96903f87ac699f3505960e744f36f7d
2018/07/19 14:58:17 mounted blob: sha256:739b3deec2edb17c512f507894c55c2681f9724191d820cdc01f668330724ca7
2018/07/19 14:58:17 mounted blob: sha256:57752e7f9593cbfb7101af994b136a369ecc8174332866622db32a264f3fbefd
2018/07/19 14:58:18 us.gcr.io/my-project/sleeper-ebdb8b8b13d4bbe1d3592de055016d37:latest: digest: sha256:6c7b96a294cad3ce613aac23c8aca5f9dd12a894354ab276c157fb5c1c2e3326 size: 592
2018/07/19 14:58:18 Published us.gcr.io/my-project/sleeper-ebdb8b8b13d4bbe1d3592de055016d37@sha256:6c7b96a294cad3ce613aac23c8aca5f9dd12a894354ab276c157fb5c1c2e3326
```
### `ko resolve`
`ko resolve` takes Kubernetes yaml files in the style of `kubectl apply`
and (based on the [model above](#the-ko-model)) determines the set of
Go import paths to build, containerize, and publish.
The output of `ko resolve` is the concatenated yaml with import paths
replaced with published image digests. Following the example above,
this would be:
```shell
# Command
export PROJECT_ID=$(gcloud config get-value core/project)
export KO_DOCKER_REPO="gcr.io/${PROJECT_ID}"
ko resolve -f deployment.yaml
# Output
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: hello-world
spec:
replicas: 1
template:
spec:
containers:
- name: hello-world
# This is the digest of the published image containing the go binary.
image: gcr.io/your-project/helloworld-badf00d@sha256:deadbeef
ports:
- containerPort: 8080
```
Some Docker Registries (e.g. gcr.io) support multi-level repository names. For
these registries, it is often useful for discoverability and provenance to
preserve the full import path, for this we expose `--preserve-import-paths`,
or `-P` for short.
```shell
# Command
export PROJECT_ID=$(gcloud config get-value core/project)
export KO_DOCKER_REPO="gcr.io/${PROJECT_ID}"
ko resolve -P -f deployment.yaml
# Output
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: hello-world
spec:
replicas: 1
template:
spec:
containers:
- name: hello-world
# This is the digest of the published image containing the go binary
# at the embedded import path.
image: gcr.io/your-project/github.com/mattmoor/examples/http/cmd/helloworld@sha256:deadbeef
ports:
- containerPort: 8080
```
It is notable that this is not the default (anymore) because certain popular
registries (including Docker Hub) do not support multi-level repository names.
### `ko apply`
`ko apply` is intended to parallel `kubectl apply`, but acts on the same
resolved output as `ko resolve` emits. It is expected that `ko apply` will act
as the vehicle for rapid iteration during development. As changes are made to a
particular application, you can run: `ko apply -f unit.yaml` to rapidly
rebuild, repush, and redeploy their changes.
`ko apply` will invoke `kubectl apply` under the covers, and therefore apply
to whatever `kubectl` context is active.
### `ko apply --watch` (EXPERIMENTAL)
The `--watch` flag (`-W` for short) does an initial `apply` as above, but as it
does, it builds up a dependency graph of your program and starts to continuously
monitor the filesystem for changes. When a file changes, it re-applies any yamls
that are affected.
For example, if I edit `github.com/foo/bar/pkg/baz/blah.go`, the tool sees that
the `github.com/foo/bar/pkg/baz` package has changed, and perhaps both
`github.com/foo/bar/cmd/one` and `github.com/foo/bar/cmd/two` consume that library
and were referenced by `config/one-deploy.yaml` and `config/two-deploy.yaml`.
The edit would effectively result in a re-application of:
```
ko apply -f config/one-deploy.yaml -f config/two-deploy.yaml
```
This flag is still experimental, and feedback is very welcome.
### `ko delete`
`ko delete` simply passes through to `kubectl delete`. It is exposed purely out
of convenience for cleaning up resources created through `ko apply`.
## With `minikube`
You can use `ko` with `minikube` via a Docker Registry, but this involves
publishing images only to pull them back down to your machine again. To avoid
this, `ko` exposes `--local` or `-L` options to instead publish the images to
the local machine's Docker daemon.
This would look something like:
```shell
# Use the minikube docker daemon.
eval $(minikube docker-env)
# Make sure minikube is the current kubectl context.
kubectl config use-context minikube
# Deploy to minikube w/o registry.
ko apply -L -f config/
# This is the same as above.
KO_DOCKER_REPO=ko.local ko apply -f config/
```
A caveat of this approach is that it will not work if your container is
configured with `imagePullPolicy: Always` because despite having the image
locally, a pull is performed to ensure we have the latest version, it still
exists, and that access hasn't been revoked. A workaround for this is to
use `imagePullPolicy: IfNotPresent`, which should work well with `ko` in
all contexts.
Images will appear in the Docker daemon as `ko.local/import.path.com/foo/cmd/bar`.
With `--local` import paths are always preserved (see `--preserve-import-paths`).
## Configuration via `.ko.yaml`
While `ko` aims to have zero configuration, there are certain scenarios where
you will want to override `ko`'s default behavior. This is done via `.ko.yaml`.
`.ko.yaml` is put into the directory from which `ko` will be invoked. One can
override the directory with the `KO_CONFIG_PATH` environment variable.
If neither is present, then `ko` will rely on its default behaviors.
### Overriding the default base image
By default, `ko` makes use of `gcr.io/distroless/base:latest` as the base image
for containers. There are a wide array of scenarios in which overriding this
makes sense, for example:
1. Pinning to a particular digest of this image for repeatable builds,
1. Replacing this streamlined base image with another with better debugging
tools (e.g. a shell, like `docker.io/library/ubuntu`).
The default base image `ko` uses can be changed by simply adding the following
line to `.ko.yaml`:
```yaml
defaultBaseImage: gcr.io/another-project/another-image@sha256:deadbeef
```
### Overriding the base for particular imports
Some of your binaries may have requirements that are a more unique, and you
may want to direct `ko` to use a particular base image for just those binaries.
The base image `ko` uses can be changed by adding the following to `.ko.yaml`:
```yaml
baseImageOverrides:
github.com/my-org/my-repo/path/to/binary: docker.io/another/base:latest
```
### Why isn't `KO_DOCKER_REPO` part of `.ko.yaml`?
Once introduced to `.ko.yaml`, you may find yourself wondering: Why does it
not hold the value of `$KO_DOCKER_REPO`?
The answer is that `.ko.yaml` is expected to sit in the root of a repository,
and get checked in and versioned alongside your source code. This also means
that the configured values will be shared across developers on a project, which
for `KO_DOCKER_REPO` is actually undesireable because each developer is (likely)
using their own docker repository and cluster.
## Including static assets
A question that often comes up after using `ko` for a while is: "How do I
include static assets in images produced with `ko`?".
For this, `ko` builds around an idiom similar to `go test` and `testdata/`.
`ko` will include all of the data under `<import path>/kodata/...` in the
images it produces.
These files are placed under `/var/run/ko/...`, but the appropriate mechanism
for referencing them should be through the `KO_DATA_PATH` environment variable.
The intent of this is to enable users to test things outside of `ko` as follows:
```shell
KO_DATA_PATH=$PWD/cmd/ko/test/kodata go run ./cmd/ko/test/*.go
2018/07/19 23:35:20 Hello there
```
This produces identical output to being run within the container locally:
```shell
ko publish -L ./cmd/ko/test
2018/07/19 23:36:11 Using base gcr.io/distroless/base:latest for github.com/google/go-containerregistry/cmd/ko/test
2018/07/19 23:36:12 Loading ko.local/github.com/google/go-containerregistry/cmd/ko/test:703c205bf2f405af520b40536b87aafadcf181562b8faa6690fd2992084c8577
2018/07/19 23:36:13 Loaded ko.local/github.com/google/go-containerregistry/cmd/ko/test:703c205bf2f405af520b40536b87aafadcf181562b8faa6690fd2992084c8577
docker run -ti --rm ko.local/github.com/google/go-containerregistry/cmd/ko/test:703c205bf2f405af520b40536b87aafadcf181562b8faa6690fd2992084c8577
2018/07/19 23:36:25 Hello there
```
... or on cluster:
```shell
ko apply -f cmd/ko/test/test.yaml
2018/07/19 23:38:24 Using base gcr.io/distroless/base:latest for github.com/google/go-containerregistry/cmd/ko/test
2018/07/19 23:38:25 Publishing us.gcr.io/my-project/test-294a7bdc57d85dc6ddeef5ba38a59fe9:latest
2018/07/19 23:38:26 mounted blob: sha256:988abcba36b5948da8baa1e3616b94c0b56da814b8f6ff3ae3ac316e375e093a
2018/07/19 23:38:26 mounted blob: sha256:57752e7f9593cbfb7101af994b136a369ecc8174332866622db32a264f3fbefd
2018/07/19 23:38:26 mounted blob: sha256:f24d43c24e22298ed99ea125af6c1b828ae07716968f78cb6d09d4291a13f2d3
2018/07/19 23:38:26 mounted blob: sha256:7a7bafbc2ae1bf844c47b33025dd459913a3fece0a94b1f3ced860675be2b79c
2018/07/19 23:38:27 us.gcr.io/my-project/test-294a7bdc57d85dc6ddeef5ba38a59fe9:latest: digest: sha256:703c205bf2f405af520b40536b87aafadcf181562b8faa6690fd2992084c8577 size: 751
2018/07/19 23:38:27 Published us.gcr.io/my-project/test-294a7bdc57d85dc6ddeef5ba38a59fe9@sha256:703c205bf2f405af520b40536b87aafadcf181562b8faa6690fd2992084c8577
pod/kodata created
kubectl logs kodata
2018/07/19 23:38:29 Hello there
```
## Relevance to Release Management
`ko` is also useful for helping manage releases. For example, if your project
periodically releases a set of images and configuration to launch those images
on a Kubernetes cluster, release binaries may be published and the configuration
generated via:
```shell
export PROJECT_ID=<YOUR RELEASE PROJECT>
export KO_DOCKER_REPO="gcr.io/${PROJECT_ID}"
ko resolve -f config/ > release.yaml
```
> Note that in this context it is recommended that you also provide `-P`, if
> supported by your Docker registry. This improves users' ability to tie release
> binaries back to their source.
This will publish all of the binary components as container images to
`gcr.io/my-releases/...` and create a `release.yaml` file containing all of the
configuration for your application with inlined image references.
This resulting configuration may then be installed onto Kubernetes clusters via:
```shell
kubectl apply -f release.yaml
```
## Acknowledgements
This work is based heavily on learnings from having built the
[Docker](https://github.com/bazelbuild/rules_docker) and
[Kubernetes](https://github.com/bazelbuild/rules_k8s) support for
[Bazel](https://bazel.build). That work was presented
[here](https://www.youtube.com/watch?v=RS1aiQqgUTA).

30
cmd/binary.go Normal file
View File

@@ -0,0 +1,30 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package main
import (
"github.com/spf13/cobra"
)
// BinaryOptions represents options for the ko binary.
type BinaryOptions struct {
// Path is the import path of the binary to publish.
Path string
}
func addImageArg(cmd *cobra.Command, lo *BinaryOptions) {
cmd.Flags().StringVarP(&lo.Path, "image", "i", lo.Path,
"The import path of the binary to publish.")
}

284
cmd/commands.go Normal file
View File

@@ -0,0 +1,284 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package main
import (
"log"
"os"
"os/exec"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
"k8s.io/kubernetes/pkg/kubectl/genericclioptions"
)
// runCmd is suitable for use with cobra.Command's Run field.
type runCmd func(*cobra.Command, []string)
// passthru returns a runCmd that simply passes our CLI arguments
// through to a binary named command.
func passthru(command string) runCmd {
return func(_ *cobra.Command, _ []string) {
// Start building a command line invocation by passing
// through our arguments to command's CLI.
cmd := exec.Command(command, os.Args[1:]...)
// Pass through our environment
cmd.Env = os.Environ()
// Pass through our stdfoo
cmd.Stderr = os.Stderr
cmd.Stdout = os.Stdout
cmd.Stdin = os.Stdin
// Run it.
if err := cmd.Run(); err != nil {
log.Fatalf("error executing %q command with args: %v; %v", command, os.Args[1:], err)
}
}
}
// addKubeCommands augments our CLI surface with a passthru delete command, and an apply
// command that realizes the promise of ko, as outlined here:
// https://github.com/google/go-containerregistry/issues/80
func addKubeCommands(topLevel *cobra.Command) {
topLevel.AddCommand(&cobra.Command{
Use: "delete",
Short: `See "kubectl help delete" for detailed usage.`,
Run: passthru("kubectl"),
// We ignore unknown flags to avoid importing everything Go exposes
// from our commands.
FParseErrWhitelist: cobra.FParseErrWhitelist{
UnknownFlags: true,
},
})
koApplyFlags := []string{}
lo := &LocalOptions{}
bo := &BinaryOptions{}
no := &NameOptions{}
fo := &FilenameOptions{}
ta := &TagsOptions{}
apply := &cobra.Command{
Use: "apply -f FILENAME",
Short: "Apply the input files with image references resolved to built/pushed image digests.",
Long: `This sub-command finds import path references within the provided files, builds them into Go binaries, containerizes them, publishes them, and then feeds the resulting yaml into "kubectl apply".`,
Example: `
# Build and publish import path references to a Docker
# Registry as:
# ${KO_DOCKER_REPO}/<package name>-<hash of import path>
# Then, feed the resulting yaml into "kubectl apply".
# When KO_DOCKER_REPO is ko.local, it is the same as if
# --local was passed.
ko apply -f config/
# Build and publish import path references to a Docker
# Registry preserving import path names as:
# ${KO_DOCKER_REPO}/<import path>
# Then, feed the resulting yaml into "kubectl apply".
ko apply --preserve-import-paths -f config/
# Build and publish import path references to a Docker
# daemon as:
# ko.local/<import path>
# Then, feed the resulting yaml into "kubectl apply".
ko apply --local -f config/
# Apply from stdin:
cat config.yaml | ko apply -f -`,
Args: cobra.NoArgs,
Run: func(cmd *cobra.Command, args []string) {
// Create a set of ko-specific flags to ignore when passing through
// kubectl global flags.
ignoreSet := make(map[string]struct{})
for _, s := range koApplyFlags {
ignoreSet[s] = struct{}{}
}
// Filter out ko flags from what we will pass through to kubectl.
kubectlFlags := []string{}
cmd.Flags().Visit(func(flag *pflag.Flag) {
if _, ok := ignoreSet[flag.Name]; !ok {
kubectlFlags = append(kubectlFlags, "--"+flag.Name, flag.Value.String())
}
})
// Issue a "kubectl apply" command reading from stdin,
// to which we will pipe the resolved files.
argv := []string{"apply", "-f", "-"}
argv = append(argv, kubectlFlags...)
kubectlCmd := exec.Command("kubectl", argv...)
// Pass through our environment
kubectlCmd.Env = os.Environ()
// Pass through our std{out,err} and make our resolved buffer stdin.
kubectlCmd.Stderr = os.Stderr
kubectlCmd.Stdout = os.Stdout
// Wire up kubectl stdin to resolveFilesToWriter.
stdin, err := kubectlCmd.StdinPipe()
if err != nil {
log.Fatalf("error piping to 'kubectl apply': %v", err)
}
go resolveFilesToWriter(fo, no, lo, ta, stdin)
// Run it.
if err := kubectlCmd.Run(); err != nil {
log.Fatalf("error executing 'kubectl apply': %v", err)
}
},
}
addLocalArg(apply, lo)
addNamingArgs(apply, no)
addFileArg(apply, fo)
addTagsArg(apply, ta)
// Collect the ko-specific apply flags before registering the kubectl global
// flags so that we can ignore them when passing kubectl global flags through
// to kubectl.
apply.Flags().VisitAll(func(flag *pflag.Flag) {
koApplyFlags = append(koApplyFlags, flag.Name)
})
// Register the kubectl global flags.
kubeConfigFlags := genericclioptions.NewConfigFlags()
kubeConfigFlags.AddFlags(apply.Flags())
topLevel.AddCommand(apply)
resolve := &cobra.Command{
Use: "resolve -f FILENAME",
Short: "Print the input files with image references resolved to built/pushed image digests.",
Long: `This sub-command finds import path references within the provided files, builds them into Go binaries, containerizes them, publishes them, and prints the resulting yaml.`,
Example: `
# Build and publish import path references to a Docker
# Registry as:
# ${KO_DOCKER_REPO}/<package name>-<hash of import path>
# When KO_DOCKER_REPO is ko.local, it is the same as if
# --local and --preserve-import-paths were passed.
ko resolve -f config/
# Build and publish import path references to a Docker
# Registry preserving import path names as:
# ${KO_DOCKER_REPO}/<import path>
# When KO_DOCKER_REPO is ko.local, it is the same as if
# --local was passed.
ko resolve --preserve-import-paths -f config/
# Build and publish import path references to a Docker
# daemon as:
# ko.local/<import path>
# This always preserves import paths.
ko resolve --local -f config/`,
Args: cobra.NoArgs,
Run: func(cmd *cobra.Command, args []string) {
resolveFilesToWriter(fo, no, lo, ta, os.Stdout)
},
}
addLocalArg(resolve, lo)
addNamingArgs(resolve, no)
addFileArg(resolve, fo)
addTagsArg(resolve, ta)
topLevel.AddCommand(resolve)
publish := &cobra.Command{
Use: "publish IMPORTPATH...",
Short: "Build and publish container images from the given importpaths.",
Long: `This sub-command builds the provided import paths into Go binaries, containerizes them, and publishes them.`,
Example: `
# Build and publish import path references to a Docker
# Registry as:
# ${KO_DOCKER_REPO}/<package name>-<hash of import path>
# When KO_DOCKER_REPO is ko.local, it is the same as if
# --local and --preserve-import-paths were passed.
ko publish github.com/foo/bar/cmd/baz github.com/foo/bar/cmd/blah
# Build and publish a relative import path as:
# ${KO_DOCKER_REPO}/<package name>-<hash of import path>
# When KO_DOCKER_REPO is ko.local, it is the same as if
# --local and --preserve-import-paths were passed.
ko publish ./cmd/blah
# Build and publish a relative import path as:
# ${KO_DOCKER_REPO}/<import path>
# When KO_DOCKER_REPO is ko.local, it is the same as if
# --local was passed.
ko publish --preserve-import-paths ./cmd/blah
# Build and publish import path references to a Docker
# daemon as:
# ko.local/<import path>
# This always preserves import paths.
ko publish --local github.com/foo/bar/cmd/baz github.com/foo/bar/cmd/blah`,
Args: cobra.MinimumNArgs(1),
Run: func(_ *cobra.Command, args []string) {
publishImages(args, no, lo, ta)
},
}
addLocalArg(publish, lo)
addNamingArgs(publish, no)
addTagsArg(publish, ta)
topLevel.AddCommand(publish)
run := &cobra.Command{
Use: "run NAME --image=IMPORTPATH",
Short: "A variant of `kubectl run` that containerizes IMPORTPATH first.",
Long: `This sub-command combines "ko publish" and "kubectl run" to support containerizing and running Go binaries on Kubernetes in a single command.`,
Example: `
# Publish the --image and run it on Kubernetes as:
# ${KO_DOCKER_REPO}/<package name>-<hash of import path>
# When KO_DOCKER_REPO is ko.local, it is the same as if
# --local and --preserve-import-paths were passed.
ko run foo --image=github.com/foo/bar/cmd/baz
# This supports relative import paths as well.
ko run foo --image=./cmd/baz`,
Run: func(cmd *cobra.Command, args []string) {
imgs := publishImages([]string{bo.Path}, no, lo, ta)
// There's only one, but this is the simple way to access the
// reference since the import path may have been qualified.
for k, v := range imgs {
log.Printf("Running %q", k)
// Issue a "kubectl run" command with our same arguments,
// but supply a second --image to override the one we intercepted.
argv := append(os.Args[1:], "--image", v.String())
kubectlCmd := exec.Command("kubectl", argv...)
// Pass through our environment
kubectlCmd.Env = os.Environ()
// Pass through our std*
kubectlCmd.Stderr = os.Stderr
kubectlCmd.Stdout = os.Stdout
kubectlCmd.Stdin = os.Stdin
// Run it.
if err := kubectlCmd.Run(); err != nil {
log.Fatalf("error executing \"kubectl run\": %v", err)
}
}
},
// We ignore unknown flags to avoid importing everything Go exposes
// from our commands.
FParseErrWhitelist: cobra.FParseErrWhitelist{
UnknownFlags: true,
},
}
addLocalArg(run, lo)
addNamingArgs(run, no)
addImageArg(run, bo)
addTagsArg(run, ta)
topLevel.AddCommand(run)
}

92
cmd/config.go Normal file
View File

@@ -0,0 +1,92 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package main
import (
"fmt"
"log"
"os"
"strconv"
"time"
"github.com/spf13/viper"
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/remote"
)
var (
defaultBaseImage name.Reference
baseImageOverrides map[string]name.Reference
)
func getBaseImage(s string) (v1.Image, error) {
ref, ok := baseImageOverrides[s]
if !ok {
ref = defaultBaseImage
}
log.Printf("Using base %s for %s", ref, s)
return remote.Image(ref, remote.WithAuthFromKeychain(authn.DefaultKeychain))
}
func getCreationTime() (*v1.Time, error) {
epoch := os.Getenv("SOURCE_DATE_EPOCH")
if epoch == "" {
return nil, nil
}
seconds, err := strconv.ParseInt(epoch, 10, 64)
if err != nil {
return nil, fmt.Errorf("the environment variable SOURCE_DATE_EPOCH is invalid. It's must be a number of seconds since January 1st 1970, 00:00 UTC, got %v", err)
}
return &v1.Time{time.Unix(seconds, 0)}, nil
}
func init() {
// If omitted, use this base image.
viper.SetDefault("defaultBaseImage", "gcr.io/distroless/static:latest")
viper.SetConfigName(".ko") // .yaml is implicit
if override := os.Getenv("KO_CONFIG_PATH"); override != "" {
viper.AddConfigPath(override)
}
viper.AddConfigPath("./")
if err := viper.ReadInConfig(); err != nil {
if _, ok := err.(viper.ConfigFileNotFoundError); !ok {
log.Fatalf("error reading config file: %v", err)
}
}
ref := viper.GetString("defaultBaseImage")
dbi, err := name.ParseReference(ref, name.WeakValidation)
if err != nil {
log.Fatalf("'defaultBaseImage': error parsing %q as image reference: %v", ref, err)
}
defaultBaseImage = dbi
baseImageOverrides = make(map[string]name.Reference)
overrides := viper.GetStringMapString("baseImageOverrides")
for k, v := range overrides {
bi, err := name.ParseReference(v, name.WeakValidation)
if err != nil {
log.Fatalf("'baseImageOverrides': error parsing %q as image reference: %v", v, err)
}
baseImageOverrides[k] = bi
}
}

133
cmd/filestuff.go Normal file
View File

@@ -0,0 +1,133 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package main
import (
"log"
"os"
"path/filepath"
"github.com/fsnotify/fsnotify"
"github.com/spf13/cobra"
)
// FilenameOptions is from pkg/kubectl.
type FilenameOptions struct {
Filenames []string
Recursive bool
Watch bool
}
func addFileArg(cmd *cobra.Command, fo *FilenameOptions) {
// From pkg/kubectl
cmd.Flags().StringSliceVarP(&fo.Filenames, "filename", "f", fo.Filenames,
"Filename, directory, or URL to files to use to create the resource")
cmd.Flags().BoolVarP(&fo.Recursive, "recursive", "R", fo.Recursive,
"Process the directory used in -f, --filename recursively. Useful when you want to manage related manifests organized within the same directory.")
cmd.Flags().BoolVarP(&fo.Watch, "watch", "W", fo.Watch,
"Continuously monitor the transitive dependencies of the passed yaml files, and redeploy whenever anything changes.")
}
// Based heavily on pkg/kubectl
func enumerateFiles(fo *FilenameOptions) chan string {
files := make(chan string)
go func() {
// When we're done enumerating files, close the channel
defer close(files)
// When we are in --watch mode, we set up watches on the filesystem locations
// that we are supplied and continuously stream files, until we are sent an
// interrupt.
var watcher *fsnotify.Watcher
if fo.Watch {
var err error
watcher, err = fsnotify.NewWatcher()
if err != nil {
log.Fatalf("Unexpected error initializing fsnotify: %v", err)
}
defer watcher.Close()
}
for _, paths := range fo.Filenames {
// Just pass through '-' as it is indicative of stdin.
if paths == "-" {
files <- paths
continue
}
// For each of the "filenames" we are passed (file or directory) start a
// "Walk" to enumerate all of the contained files recursively.
err := filepath.Walk(paths, func(path string, fi os.FileInfo, err error) error {
if err != nil {
return err
}
// If this is a directory, skip it if it isn't the current directory we are
// processing (unless we are in recursive mode). If we decide to process
// the directory, and we're in watch mode, then we set up a watch on the
// directory.
if fi.IsDir() {
if path != paths && !fo.Recursive {
return filepath.SkipDir
}
if watcher != nil {
watcher.Add(path)
}
// We don't stream back directories, we just decide to skip them, or not.
return nil
}
// Don't check extension if the filepath was passed explicitly
if path != paths {
switch filepath.Ext(path) {
case ".json", ".yaml":
// Process these.
default:
return nil
}
// We weren't passed this explicitly, so elide the watch as we
// are already watching the directory.
} else {
// We were passed this directly, and so we may not be watching the
// directory, so watch this file explicitly.
if watcher != nil {
watcher.Add(path)
}
}
files <- path
return nil
})
if err != nil {
log.Fatalf("Error enumerating files: %v", err)
}
}
// We're done watching the files we were passed and setting up watches.
// Now listen for change events from the watches we set up and resend
// files that change as if we just saw them (so they can be reprocessed).
if watcher != nil {
for {
select {
case event := <-watcher.Events:
switch filepath.Ext(event.Name) {
case ".json", ".yaml":
files <- event.Name
}
case err := <-watcher.Errors:
log.Fatalf("Error watching: %v", err)
}
}
}
}()
return files
}

52
cmd/flatname.go Normal file
View File

@@ -0,0 +1,52 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package main
import (
"crypto/md5"
"encoding/hex"
"path/filepath"
"github.com/spf13/cobra"
)
// NameOptions represents options for the ko binary.
type NameOptions struct {
// PreserveImportPaths preserves the full import path after KO_DOCKER_REPO.
PreserveImportPaths bool
// BaseImportPaths uses the base path without MD5 hash after KO_DOCKER_REPO.
BaseImportPaths bool
}
func addNamingArgs(cmd *cobra.Command, no *NameOptions) {
cmd.Flags().BoolVarP(&no.PreserveImportPaths, "preserve-import-paths", "P", no.PreserveImportPaths,
"Whether to preserve the full import path after KO_DOCKER_REPO.")
cmd.Flags().BoolVarP(&no.BaseImportPaths, "base-import-paths", "B", no.BaseImportPaths,
"Whether to use the base path without MD5 hash after KO_DOCKER_REPO.")
}
func packageWithMD5(importpath string) string {
hasher := md5.New()
hasher.Write([]byte(importpath))
return filepath.Base(importpath) + "-" + hex.EncodeToString(hasher.Sum(nil))
}
func preserveImportPath(importpath string) string {
return importpath
}
func baseImportPaths(importpath string) string {
return filepath.Base(importpath)
}

30
cmd/local.go Normal file
View File

@@ -0,0 +1,30 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package main
import (
"github.com/spf13/cobra"
)
// LocalOptions represents options for the ko binary.
type LocalOptions struct {
// Local publishes images to a local docker daemon.
Local bool
}
func addLocalArg(cmd *cobra.Command, lo *LocalOptions) {
cmd.Flags().BoolVarP(&lo.Local, "local", "L", lo.Local,
"Whether to publish images to a local docker daemon vs. a registry.")
}

37
cmd/main.go Normal file
View File

@@ -0,0 +1,37 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package main
import (
"log"
"github.com/spf13/cobra"
)
func main() {
// Parent command to which all subcommands are added.
cmds := &cobra.Command{
Use: "ko",
Short: "Rapidly iterate with Go, Containers, and Kubernetes.",
Run: func(cmd *cobra.Command, args []string) {
cmd.Help()
},
}
addKubeCommands(cmds)
if err := cmds.Execute(); err != nil {
log.Fatalf("error during command execution: %v", err)
}
}

106
cmd/publish.go Normal file
View File

@@ -0,0 +1,106 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package main
import (
"fmt"
gb "go/build"
"log"
"os"
"path/filepath"
"strings"
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/ko/pkg/build"
"github.com/google/ko/pkg/publish"
)
func qualifyLocalImport(importpath, gopathsrc, pwd string) (string, error) {
if !strings.HasPrefix(pwd, gopathsrc) {
return "", fmt.Errorf("pwd (%q) must be on $GOPATH/src (%q) to support local imports", pwd, gopathsrc)
}
// Given $GOPATH/src and $PWD (which must be within $GOPATH/src), trim
// off $GOPATH/src/ from $PWD and append local importpath to get the
// fully-qualified importpath.
return filepath.Join(strings.TrimPrefix(pwd, gopathsrc+string(filepath.Separator)), importpath), nil
}
func publishImages(importpaths []string, no *NameOptions, lo *LocalOptions, ta *TagsOptions) map[string]name.Reference {
opt, err := gobuildOptions()
if err != nil {
log.Fatalf("error setting up builder options: %v", err)
}
b, err := build.NewGo(opt...)
if err != nil {
log.Fatalf("error creating go builder: %v", err)
}
imgs := make(map[string]name.Reference)
for _, importpath := range importpaths {
if gb.IsLocalImport(importpath) {
// Qualify relative imports to their fully-qualified
// import path, assuming $PWD is within $GOPATH/src.
gopathsrc := filepath.Join(gb.Default.GOPATH, "src")
pwd, err := os.Getwd()
if err != nil {
log.Fatalf("error getting current working directory: %v", err)
}
importpath, err = qualifyLocalImport(importpath, gopathsrc, pwd)
if err != nil {
log.Fatal(err)
}
}
if !b.IsSupportedReference(importpath) {
log.Fatalf("importpath %q is not supported", importpath)
}
img, err := b.Build(importpath)
if err != nil {
log.Fatalf("error building %q: %v", importpath, err)
}
var pub publish.Interface
repoName := os.Getenv("KO_DOCKER_REPO")
var namer publish.Namer
if no.PreserveImportPaths {
namer = preserveImportPath
} else if no.BaseImportPaths {
namer = baseImportPaths
} else {
namer = packageWithMD5
}
if lo.Local || repoName == publish.LocalDomain {
pub = publish.NewDaemon(namer, ta.Tags)
} else {
if _, err := name.NewRepository(repoName, name.WeakValidation); err != nil {
log.Fatalf("the environment variable KO_DOCKER_REPO must be set to a valid docker repository, got %v", err)
}
opts := []publish.Option{publish.WithAuthFromKeychain(authn.DefaultKeychain), publish.WithNamer(namer), publish.WithTags(ta.Tags)}
pub, err = publish.NewDefault(repoName, opts...)
if err != nil {
log.Fatalf("error setting up default image publisher: %v", err)
}
}
ref, err := pub.Publish(img, importpath)
if err != nil {
log.Fatalf("error publishing %s: %v", importpath, err)
}
imgs[importpath] = ref
}
return imgs
}

48
cmd/publish_test.go Normal file
View File

@@ -0,0 +1,48 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package main
import "testing"
func TestQualifyLocalImport(t *testing.T) {
for _, c := range []struct {
importpath, gopathsrc, pwd, want string
wantErr bool
}{{
importpath: "./cmd/foo",
gopathsrc: "/home/go/src",
pwd: "/home/go/src/github.com/my/repo",
want: "github.com/my/repo/cmd/foo",
}, {
importpath: "./foo",
gopathsrc: "/home/go/src",
pwd: "/home/go/src/github.com/my/repo/cmd",
want: "github.com/my/repo/cmd/foo",
}, {
// $PWD not on $GOPATH/src
importpath: "./cmd/foo",
gopathsrc: "/home/go/src",
pwd: "/",
wantErr: true,
}} {
got, err := qualifyLocalImport(c.importpath, c.gopathsrc, c.pwd)
if gotErr := err != nil; gotErr != c.wantErr {
t.Fatalf("qualifyLocalImport returned %v, wanted err? %t", err, c.wantErr)
}
if got != c.want {
t.Fatalf("qualifyLocalImport(%q, %q, %q): got %q, want %q", c.importpath, c.gopathsrc, c.pwd, got, c.want)
}
}
}

259
cmd/resolve.go Normal file
View File

@@ -0,0 +1,259 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package main
import (
"fmt"
"io"
"io/ioutil"
"log"
"os"
"sync"
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"
"github.com/mattmoor/dep-notify/pkg/graph"
"github.com/google/ko/pkg/build"
"github.com/google/ko/pkg/publish"
"github.com/google/ko/pkg/resolve"
)
func gobuildOptions() ([]build.Option, error) {
creationTime, err := getCreationTime()
if err != nil {
return nil, err
}
opts := []build.Option{
build.WithBaseImages(getBaseImage),
}
if creationTime != nil {
opts = append(opts, build.WithCreationTime(*creationTime))
}
return opts, nil
}
func makeBuilder() (*build.Caching, error) {
opt, err := gobuildOptions()
if err != nil {
log.Fatalf("error setting up builder options: %v", err)
}
innerBuilder, err := build.NewGo(opt...)
if err != nil {
return nil, err
}
// tl;dr Wrap builder in a caching builder.
//
// The caching builder should on Build calls:
// - Check for a valid Build future
// - if a valid Build future exists at the time of the request,
// then block on it.
// - if it does not, then initiate and record a Build future.
// - When import paths are "affected" by filesystem changes during a
// Watch, then invalidate their build futures *before* we put the
// affected yaml files onto the channel
//
// This will benefit the following key cases:
// 1. When the same import path is referenced across multiple yaml files
// we can elide subsequent builds by blocking on the same image future.
// 2. When an affected yaml file has multiple import paths (mostly unaffected)
// we can elide the builds of unchanged import paths.
return build.NewCaching(innerBuilder)
}
func makePublisher(no *NameOptions, lo *LocalOptions, ta *TagsOptions) (publish.Interface, error) {
// Create the publish.Interface that we will use to publish image references
// to either a docker daemon or a container image registry.
innerPublisher, err := func() (publish.Interface, error) {
namer := func() publish.Namer {
if no.PreserveImportPaths {
return preserveImportPath
}
return packageWithMD5
}()
repoName := os.Getenv("KO_DOCKER_REPO")
if lo.Local || repoName == publish.LocalDomain {
return publish.NewDaemon(namer, ta.Tags), nil
}
_, err := name.NewRepository(repoName, name.WeakValidation)
if err != nil {
return nil, fmt.Errorf("the environment variable KO_DOCKER_REPO must be set to a valid docker repository, got %v", err)
}
return publish.NewDefault(repoName,
publish.WithAuthFromKeychain(authn.DefaultKeychain),
publish.WithNamer(namer),
publish.WithTags(ta.Tags))
}()
if err != nil {
return nil, err
}
// Wrap publisher in a memoizing publisher implementation.
return publish.NewCaching(innerPublisher)
}
// resolvedFuture represents a "future" for the bytes of a resolved file.
type resolvedFuture chan []byte
func resolveFilesToWriter(fo *FilenameOptions, no *NameOptions, lo *LocalOptions, ta *TagsOptions, out io.WriteCloser) {
defer out.Close()
builder, err := makeBuilder()
if err != nil {
log.Fatalf("error creating builder: %v", err)
}
// Wrap publisher in a memoizing publisher implementation.
publisher, err := makePublisher(no, lo, ta)
if err != nil {
log.Fatalf("error creating publisher: %v", err)
}
// By having this as a channel, we can hook this up to a filesystem
// watcher and leave `fs` open to stream the names of yaml files
// affected by code changes (including the modification of existing or
// creation of new yaml files).
fs := enumerateFiles(fo)
// This tracks filename -> []importpath
var sm sync.Map
var g graph.Interface
var errCh chan error
if fo.Watch {
// Start a dep-notify process that on notifications scans the
// file-to-recorded-build map and for each affected file resends
// the filename along the channel.
g, errCh, err = graph.New(func(ss graph.StringSet) {
sm.Range(func(k, v interface{}) bool {
key := k.(string)
value := v.([]string)
for _, ip := range value {
if ss.Has(ip) {
// See the comment above about how "builder" works.
builder.Invalidate(ip)
fs <- key
}
}
return true
})
})
if err != nil {
log.Fatalf("Error creating dep-notify graph: %v", err)
}
// Cleanup the fsnotify hooks when we're done.
defer g.Shutdown()
}
var futures []resolvedFuture
for {
// Each iteration, if there is anything in the list of futures,
// listen to it in addition to the file enumerating channel.
// A nil channel is never available to receive on, so if nothing
// is available, this will result in us exclusively selecting
// on the file enumerating channel.
var bf resolvedFuture
if len(futures) > 0 {
bf = futures[0]
} else if fs == nil {
// There are no more files to enumerate and the futures
// have been drained, so quit.
break
}
select {
case f, ok := <-fs:
if !ok {
// a nil channel is never available to receive on.
// This allows us to drain the list of in-process
// futures without this case of the select winning
// each time.
fs = nil
break
}
// Make a new future to use to ship the bytes back and append
// it to the list of futures (see comment below about ordering).
ch := make(resolvedFuture)
futures = append(futures, ch)
// Kick off the resolution that will respond with its bytes on
// the future.
go func(f string) {
defer close(ch)
// Record the builds we do via this builder.
recordingBuilder := &build.Recorder{
Builder: builder,
}
b, err := resolveFile(f, recordingBuilder, publisher)
if err != nil {
// Don't let build errors disrupt the watch.
lg := log.Fatalf
if fo.Watch {
lg = log.Printf
}
lg("error processing import paths in %q: %v", f, err)
return
}
// Associate with this file the collection of binary import paths.
sm.Store(f, recordingBuilder.ImportPaths)
ch <- b
if fo.Watch {
for _, ip := range recordingBuilder.ImportPaths {
// Technically we never remove binary targets from the graph,
// which will increase our graph's watch load, but the
// notifications that they change will result in no affected
// yamls, and no new builds or deploys.
if err := g.Add(ip); err != nil {
log.Fatalf("Error adding importpath to dep graph: %v", err)
}
}
}
}(f)
case b, ok := <-bf:
// Once the head channel returns something, dequeue it.
// We listen to the futures in order to be respectful of
// the kubectl apply ordering, which matters!
futures = futures[1:]
if ok {
// Write the next body and a trailing delimiter.
// We write the delimeter LAST so that when streamed to
// kubectl it knows that the resource is complete and may
// be applied.
out.Write(append(b, []byte("\n---\n")...))
}
case err := <-errCh:
log.Fatalf("Error watching dependencies: %v", err)
}
}
}
func resolveFile(f string, builder build.Interface, pub publish.Interface) (b []byte, err error) {
if f == "-" {
b, err = ioutil.ReadAll(os.Stdin)
} else {
b, err = ioutil.ReadFile(f)
}
if err != nil {
return nil, err
}
return resolve.ImageReferences(b, builder, pub)
}

29
cmd/tags.go Normal file
View File

@@ -0,0 +1,29 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package main
import (
"github.com/spf13/cobra"
)
// TagsOptions holds the list of tags to tag the built image
type TagsOptions struct {
Tags []string
}
func addTagsArg(cmd *cobra.Command, ta *TagsOptions) {
cmd.Flags().StringSliceVarP(&ta.Tags, "tags", "t", []string{"latest"},
"Which tags to use for the produced image instead of the default 'latest' tag.")
}

1
cmd/test/kenobi Normal file
View File

@@ -0,0 +1 @@
Hello there

1
cmd/test/kodata/kenobi Symbolic link
View File

@@ -0,0 +1 @@
../kenobi

32
cmd/test/main.go Normal file
View File

@@ -0,0 +1,32 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package main
import (
"io/ioutil"
"log"
"os"
"path/filepath"
)
func main() {
dp := os.Getenv("KO_DATA_PATH")
file := filepath.Join(dp, "kenobi")
bytes, err := ioutil.ReadFile(file)
if err != nil {
log.Fatalf("Error reading %q: %v", file, err)
}
log.Printf(string(bytes))
}

24
cmd/test/test.yaml Normal file
View File

@@ -0,0 +1,24 @@
# Copyright 2018 Google LLC All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: v1
kind: Pod
metadata:
name: kodata
annotations:
sidecar.istio.io/inject: "false"
spec:
containers:
- name: obiwan
image: github.com/google/go-containerregistry/cmd/ko/test
restartPolicy: Never