Skip to content

Conversation

@iQQBot
Copy link
Contributor

@iQQBot iQQBot commented Jan 14, 2026

Summary

Go packages with packaging: library are now automatically treated as "weak dependencies" when referenced, improving build parallelism for Go monorepos.

Motivation

In Go monorepos, libraries are typically used for their source code via go.mod replace directives, not for their built artifacts. Previously, leeway would:

  1. Build the library first (blocking)
  2. Extract the built artifact
  3. Then build the dependent package

This was inefficient because the dependent package only needs the library's source files, not its built artifact.

Changes

Weak Dependencies for Go Libraries

When a Go package with packaging: library is listed as a dependency, it's automatically converted to a "weak dependency":

Aspect Regular Dependency Go Library (Weak)
Affects package version
Must be built first
What's copied to _deps/ Built artifact Source files
go.mod replace added
Build runs Sequentially In parallel

Important Constraint

A Go library can only be a weak dependency if all its transitive dependencies are also Go libraries. If a Go library depends on non-Go-library packages (e.g., generic packages), it's treated as a regular hard dependency.

# CAN be weak dep - only depends on other Go libraries
go-lib-a:
  type: go
  config:
    packaging: library
  deps:
    - other-go-lib:lib

# MUST be hard dep - depends on a generic package  
go-lib-b:
  type: go
  config:
    packaging: library
  deps:
    - some-generic:pkg

Build Flow

Before:
1. Build lib-a (blocking)
2. Build lib-b (blocking) 
3. Build app

After (when libs have only Go library deps):
1. Start in parallel:
   ├── Build app (copies lib-a, lib-b sources to _deps/)
   ├── Build lib-a (runs tests)
   └── Build lib-b (runs tests)

Weak Dependency Failure Propagation

If a weak dependency fails (e.g., tests fail), all packages that depend on it will also fail before running their build commands. This prevents publishing packages (e.g., Docker images) when their weak dependencies have failed.

Example:
  docker:img -> go-app:app -> go-lib:lib (weak dep)
  
If go-lib tests fail:
1. go-lib:lib build fails
2. go-app:app waits for go-lib, sees failure, fails before building
3. docker:img waits for go-lib (via go-app), sees failure, fails before docker push

Implementation uses a broadcast channel pattern - multiple packages can wait on the same weak dep result.

Cache Behavior

  • Source files are always copied from the workspace (not from cache)
  • Version manifest includes all transitive weak deps
  • Any change to a library invalidates the dependent package's cache

Example

# my-lib/BUILD.yaml
packages:
- name: lib
  type: go
  config:
    packaging: library  # Automatically becomes weak dep when referenced

# my-app/BUILD.yaml  
packages:
- name: app
  type: go
  deps:
    - my-lib:lib  # Sources copied, builds in parallel
  config:
    packaging: app

Testing

  • Added unit tests for canBeWeakDep() and checkAllDepsAreGoLibraries()
  • Added unit tests for GetTransitiveWeakDependencies()
  • Added unit tests for collectWeakDependencies()
  • Added tests for auto-conversion behavior
  • Added tests for version manifest inclusion
  • All existing tests pass
  • All integration tests pass

@iQQBot iQQBot force-pushed the feature/go-library-weak-deps branch 6 times, most recently from 210c8cd to 5fc7d5c Compare January 15, 2026 07:50
@iQQBot iQQBot marked this pull request as ready for review January 15, 2026 16:36
@iQQBot iQQBot force-pushed the feature/go-library-weak-deps branch 2 times, most recently from 7da9ab1 to 69b3ee4 Compare January 15, 2026 20:42
Copy link
Member

@WVerlaek WVerlaek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

otherwise LGTM!

@iQQBot iQQBot force-pushed the feature/go-library-weak-deps branch 2 times, most recently from a38626b to 2d4d09f Compare January 19, 2026 13:47
iQQBot and others added 3 commits January 19, 2026 14:02
Go packages with packaging: library are now automatically treated as
weak dependencies when referenced. This means:

- Source files are copied to _deps/ (not built artifacts)
- go.mod replace directives are added
- Builds run in parallel (don't block dependent package)
- Version tracking ensures cache invalidation when libraries change

This improves build parallelism for Go monorepos where libraries
are used for source code via go.mod replace, not for built artifacts.

Co-authored-by: Ona <no-reply@ona.com>
A Go library can only be a weak dependency if ALL its transitive
dependencies are also Go libraries. If a Go library depends on
non-Go-library packages (e.g., generic packages), it's treated
as a regular hard dependency.

This simplifies the implementation by removing the need to handle
hard deps of weak deps separately.

Co-authored-by: Ona <no-reply@ona.com>
- Release build semaphore before waiting for weak deps (allows other builds to proceed)
- All packages wait for their weak deps (transitive failures propagate)
- Remove artifact from cache if weak dep fails
- Respect src rules when copying weak dependency sources

Co-authored-by: Ona <no-reply@ona.com>
@iQQBot iQQBot force-pushed the feature/go-library-weak-deps branch from 2d4d09f to 44e5c89 Compare January 19, 2026 14:06
The weak dependency feature is now opt-in via CLI flag.
When not enabled, weak deps are treated as hard deps (built artifacts
are copied instead of source code).

- Add GetEffectiveDependencies/GetEffectiveTransitiveDependencies helpers
- Update buildDependencies to use effective deps
- Update Build() to use effective transitive deps
- Update buildGo() to use effective deps

Co-authored-by: Ona <no-reply@ona.com>
@iQQBot iQQBot force-pushed the feature/go-library-weak-deps branch from 44e5c89 to 7dc802f Compare January 20, 2026 19:50
Tests verify that:
- With flag enabled: only hard deps are returned, weak deps handled separately
- With flag disabled: weak deps are treated as hard deps (combined)
- Transitive dependencies work correctly in both modes

Co-authored-by: Ona <no-reply@ona.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants