* Support osversion when selecting base images
Use this in e2e tests.
Also clean up usage of bo.Platform vs a new platform var.
* use correct osversion
Previously, we'd use the ctx from the Build invocation, but because
remote stashes the remote.WithContext option, when we later go to fetch
the actual base image to write it out to a tarball, it will fail because
the Build ctx was cancelled.
For whatever reason, NewGo takes a ctx, so this change propagates the
badness from remote so we can use the NewGo ctx when we fetch base
images.
In the PR where we added concurrency we used a fixed length array to store
addendum to preserve the ordering from the base image when constructing the
final index. However, with `--platform=...` this list may be filtered, which
gives us `nil` entries in our addendum.
This filters `nil` entries prior to constructing the index.
Fixes: https://github.com/google/ko/issues/544
* Build each platform concurrently
At least on my machine, this is slightly faster on a cold build and
saves significant time on a warm build.
* Plumb --jobs down to multi-arch
* Add deprecation warning for build.Limiter
This saves a roundtrip to the registry if we ever use the same base
image for multiple builds in a single invocation.
This also sets us up for using an on-disk cache for image metadata to
speed things up even further.
* Introduce KOCACHE
Cache binaries under $KOCACHE/<import path>/<platform>
Cache metdata mapping buildid to diffid and diffid to descriptor.
* Point TODO at issue
* Split layerCache into separate file
This makes things a little cleaner by having a single place that calls
buildLayer and passing a thunk down into the cache logic to call that on
a cache miss.
Also, remove the debug logging to make the code easier to follow (if you
need to recompile anyway, it's easy enough to add log lines).
* Move cached output to $KOCACHE/bin
* Use default `completion` command provided by cobra.
`cobra` provides default `completion` subcommand.
It supports bash, zsh, fish, and powershell by default.
* update docs
We were emitting package checksum hashes as `h1:{base64}`. `h1:` is a prefix that indicates "Hash 1", which is a SHA-256 based hash of the files, which is then base64 encoded as the suffix.
This change detects/strips the `h1:` prefix and re-encodes the base64 data as hex.
* Connect SBOMs with SPDX support.
This combines Jason's SPDX stuff and my SBOM stuff to support
SPDX-based SBOMs by default instead of our `go version -m`
invention.
* Make ko deps use SPDX by default
* WIP: generate ko deps in SPDX format
- copy out a bunch of BuildInfo stuff that will land in 1.18
* review comments
* have deps take --sbom flag more like Matt's new publish-time flag
This adds functionality that enables the default publisher to
publish SBOMs (and later signatures and attestations) when the
`build.Result` is an `oci.SignedEntity`.
This also changes the `gobuild` logic to start producing
`oci.Signed*` as its `build.Result`s, so when executed we get an
SBOM for each architecture image.
For example, see the "Published SBOM" lines below:
```shell
2021/11/19 19:24:50 Using base gcr.io/distroless/static:nonroot for github.com/google/ko
2021/11/19 19:24:51 Building github.com/google/ko for linux/amd64
2021/11/19 19:24:52 Building github.com/google/ko for linux/arm64
2021/11/19 19:24:57 Publishing ghcr.io/mattmoor/ko:latest
2021/11/19 19:24:58 existing blob: sha256:c78c74e7bb4a511f7d31061fbf140d55d5549a62d33cdbdf0c57ffe43603bbeb
2021/11/19 19:24:58 existing blob: sha256:4aa59d0bf53d4190174fbbfa3e9b15fdab72e5a95077025abfa8435ccafa2920
2021/11/19 19:24:58 ghcr.io/mattmoor/ko:sha256-d2bc030f5ed083d5e6a30a7969c9a8e599511b8d7a6e20695bf5ea029b6e2c3f.sbom: digest: sha256:c67ec671aaa82902e619883a7ac7486e6f9af36653449e2eb030ba273fe5a022 size: 348
2021/11/19 19:24:58 Published SBOM ghcr.io/mattmoor/ko:sha256-d2bc030f5ed083d5e6a30a7969c9a8e599511b8d7a6e20695bf5ea029b6e2c3f.sbom
2021/11/19 19:24:58 existing blob: sha256:c78c74e7bb4a511f7d31061fbf140d55d5549a62d33cdbdf0c57ffe43603bbeb
2021/11/19 19:24:58 existing blob: sha256:4aa59d0bf53d4190174fbbfa3e9b15fdab72e5a95077025abfa8435ccafa2920
2021/11/19 19:24:59 ghcr.io/mattmoor/ko:sha256-b74c230f20efd94981e5fd823bacc23cbd71055a1b3b6a0893152b398c67743b.sbom: digest: sha256:c67ec671aaa82902e619883a7ac7486e6f9af36653449e2eb030ba273fe5a022 size: 348
2021/11/19 19:24:59 Published SBOM ghcr.io/mattmoor/ko:sha256-b74c230f20efd94981e5fd823bacc23cbd71055a1b3b6a0893152b398c67743b.sbom
2021/11/19 19:24:59 existing blob: sha256:3f7e3c6765a6abc682cd40ea256fbea5c1d4debbc07659efbc0dedc13eee0da6
2021/11/19 19:24:59 existing blob: sha256:250c06f7c38e52dc77e5c7586c3e40280dc7ff9bb9007c396e06d96736cf8542
2021/11/19 19:24:59 existing blob: sha256:e8614d09b7bebabd9d8a450f44e88a8807c98a438a2ddd63146865286b132d1b
2021/11/19 19:24:59 existing blob: sha256:7067b1bc6f9ce59f3a4ed2216946ebbb27a4f7a102f55d96c6af1dc90e77b510
2021/11/19 19:25:00 ghcr.io/mattmoor/ko@sha256:d2bc030f5ed083d5e6a30a7969c9a8e599511b8d7a6e20695bf5ea029b6e2c3f: digest: sha256:d2bc030f5ed083d5e6a30a7969c9a8e599511b8d7a6e20695bf5ea029b6e2c3f size: 751
2021/11/19 19:25:01 existing blob: sha256:250c06f7c38e52dc77e5c7586c3e40280dc7ff9bb9007c396e06d96736cf8542
2021/11/19 19:25:02 pushed blob: sha256:121c637d5c84562b51404a6f71c1f995ad059740293a3911a0dc33eb223e41a4
2021/11/19 19:25:02 pushed blob: sha256:859e03b7461b2a512159493ef1504d2859ed37c05ed1ef781ff98394ea4799b5
2021/11/19 19:25:02 pushed blob: sha256:d1b55c3db0f16b5056776c6d2c279efd16d28dbf1aae3eef1f3f9b7551d1f490
2021/11/19 19:25:03 ghcr.io/mattmoor/ko@sha256:b74c230f20efd94981e5fd823bacc23cbd71055a1b3b6a0893152b398c67743b: digest: sha256:b74c230f20efd94981e5fd823bacc23cbd71055a1b3b6a0893152b398c67743b size: 751
2021/11/19 19:25:03 ghcr.io/mattmoor/ko:latest: digest: sha256:e4466a7dd9be66c7c1b43a8ecc19247041ece232407a14e3d6ea3c51d2561a71 size: 529
2021/11/19 19:25:03 Published ghcr.io/mattmoor/ko@sha256:e4466a7dd9be66c7c1b43a8ecc19247041ece232407a14e3d6ea3c51d2561a71
ghcr.io/mattmoor/ko@sha256:e4466a7dd9be66c7c1b43a8ecc19247041ece232407a14e3d6ea3c51d2561a71
```
The "SBOM" being attached in this change is the raw output of `go version -m`,
which we will convert to one of the standard formats in a subsequent change.
Enables programmatic control of whether `ko` adds the `-trimpath`
flag to `go build`.
The `-trimpath` flag removes file system paths from the resulting
binary. `ko` adds `-trimpath` by default as it aids in achieving
reproducible builds.
However, removing file system paths makes interactive debugging more
challenging, in particular in mapping source file locations in the
IDE to debug information in the binary.
If you set `Trimpath` to `false` to enable interactive debugging, you
probably also want to set `DisableOptimizations` to `true` to disable
compiler optimizations and inlining.
Reference for `-trimpath`:
https://pkg.go.dev/cmd/go#hdr-Compile_packages_and_dependenciesResolves: #500
Related: #71, #78, https://github.com/GoogleContainerTools/skaffold/issues/6843
* Use tools/go/packages in place of go/build
* Use build config dir
Signed-off-by: Ben Moss <benm@vmware.com>
* Use filepath.Dir in place of ".." for explicitness
* Add build config usage log statement
There is currently no indication whether `ko` picks one of the configured
build configurations from the `.ko.yaml` configuration file for a build.
Add log statement to print the build config being picked for the build.
Introduce default entry for build config `ID` in case it is not specified.
* Add path check for build configuration settings
Add `os.Stat` to verify that the path that is configured in the build
configuration entry is valid. As a side effect, this will print out an error
message in case someone sets an import path like `github.com/google/ko` in
the `main` field of the build config.
* Fix trimpath command line flag in README
Fixed wrong command line flag `--trimpath` to `-trimpath`.
Ensure that the directory specified in build configs in `.ko.yaml` is
used to:
1. Load module information
2. Resolve local paths to Go import paths
3. Working directory for compilation
The change achieves this by introducing `gobuilds`, which contains a
map of import path to `build.Interface` instances. Each entry maps to a
`builds` entry from `.ko.yaml`. `gobuilds` dispatches to the builder
instances based on the requested import path, and falls back to a
default builder if there's no match.
Thanks to @jonjohnsonjr for the suggestions in
https://github.com/google/ko/issues/422#issuecomment-909408527
Also removes mutable globals in the `commands` package.
Fixes: #422
* Use working directory and build config `dir`
Use the working directory from `BuildOptions` to load `.ko.yaml`.
Also, use the `dir` build config field to load package information,
instead of assuming that `go.mod` is in the current working directory.
This removes the `init()` function from `./pkg/commands/config.go`.
And avoids the global viper instance, which caused some Heisenbugs (and
associated hair loss).
Fixes: #422, #424
* Return error instead of log.Fatal
`log.Fatal` is no longer needed in `loadConfig()`, since it's no longer
an `init()` function.
Also removed `log.Fatal` from `createBuildConfigMap()`.
* Set build config via BuildOptions
Enables programmatically overriding build configs when ko is
embedded in another tool.
Related: #340, #419
* Use local registry for base images in unit tests
Tests create a local registry (using ggcr) with a dummy base image. This
speeds up tests, since they don't need to hit gcr.io to fetch the
default distroless base image.
* Update function comment to refer to random image
* Generate Markdown docs
This is largely copied from similar work in go-containerregistry
This required moving the Root command definition out of main() into a
place where it could be referenced from the gendoc tooling.
* fix boilerplate
* moar fix boilerplate
* update cmd/ko/main.go
* set -j to GOMAXPROCS at runtime
* rebase on cli-runtime change
* remove trailing whitespace
* Don't set image.base.name if base is specified by digest
* don't set empty annotation
* annotate Results, not Images and Indexes separately
* moar cleanup
* skip annotations check for images in indexes, these won't be annotated anymore
* first pass: kubectl flags must be passed after '--'
* add warning when using non-separated flags
* mark flags as deprecated
* drop defaultCacheDir and homedir dependency
* Implement ko deps
* actually add deps.go
* specify auth, useragent, platform
* stop reading tar if the context is cancelled
* chmod to the file's perms
* remove support for --platform, modules don't care about build tags
* fix copyright boilerplate
* drop fs dependency
* udpate module integration test to newer Go versions
* use entrypoint to identify the binary
* fix gosec finding, some style comments
* revert modules integration test change
* Build working Windows container images
Add e2e tests that run on Windows and cover kodata behavior
* now successfully skipping symlinks on windows :-/
* fix e2e test on windows, that relied on a symlink in kodata after all
* document windows symlink issue
* review feedback
* re-add kodata symlink tests for linux
