README
¶
Golang System Components
This library provides a set of types and functions to help enterprise developers describe, deploy, and test the Golang components that build up their software system.
Describing components
The basic building block of a backend system is a component. A component is a deployable piece of message-driven code with the capabilities to observe and react to events in an enterprise's ecosystem.
Each component has the potential to notify other components about changes to its exposed Aspect. We say that those components have an Interest to track the component's Aspect. As such, a component may have none, one or many Aspects and Interests.
Acknowledgments
Special thanks to @ofektavor, @yuvalmendelovich, @marombracha, and @arieltod for your contributions and collaboration on this project.
A very special thank you to @sgebbie; many of the concepts in this project came from working together and drawing from your deep experience in systems engineering.
A very special thank you to @tal-shani for being there from the beginning. Your unwavering support, thoughtful guidance, and genuine partnership have been invaluable throughout this journey.
Working with you all was a pleasure, and your fingerprints are all over the good parts of this codebase. Cheers!
Documentation
¶
Index ¶
- Variables
- func ExtractDoc(content, name string) (string, error)
- func InjectLogger(ctx context.Context, logger *slog.Logger) context.Context
- func Logger(ctx context.Context) *slog.Logger
- func MergeTargets(t ...[]string) (targets []string)
- func MustExtractDoc(content, name string) string
- func Run(body Procedure, opts ...Option)
- func RunProc(main Proc, opts ...Option)
- type CleanupFunc
- type Descriptor
- type ForkOption
- type L
- func (l *L) Cleanup(fn func())
- func (l *L) CleanupBackground(fn func(context.Context) error)
- func (l *L) CleanupContext(fn func(context.Context) error)
- func (l *L) CleanupError(fn func() error)
- func (l *L) Context() context.Context
- func (l *L) Continue() bool
- func (l *L) Done() <-chan struct{}
- func (l *L) Error(err error)
- func (l *L) Errorf(format string, a ...any)
- func (l *L) Fatal(err error)
- func (l *L) Fatalf(format string, a ...any)
- func (l *L) Fork(name string, procedure Procedure, opts ...ForkOption)
- func (l *L) ForkE(name string, proc ProcE)
- func (l *L) Go(name string, proc Proc)
- func (l *L) GraceContext() context.Context
- func (l *L) Log(args ...any)
- func (l *L) Logf(format string, args ...any)
- func (l *L) Name() string
- func (l *L) Stop(timeout time.Duration) (stopped bool)
- func (l *L) Stopping() <-chan struct{}
- func (l *L) Terminate()
- type Linker
- type Option
- func OnComplete(hook func(name string)) Option
- func OnStarted(hook func(name string)) Option
- func WithCompletion(done chan struct{}) Option
- func WithContext(ctx context.Context) Option
- func WithForkSpanName(name string) Option
- func WithLogger(logger *log.Logger) Option
- func WithName(name string) Option
- func WithSpan(name string) Option
- func WithStopper(stopper <-chan struct{}) Option
- type Proc
- type ProcE
- type Procedure
Constants ¶
This section is empty.
Variables ¶
var ErrStopped = errors.New("component gracefully stopped")
ErrStopped is the cause of L.GraceContext() cancellation due to a call to L.Stop(). It indicates that the component is in the process of stopping gracefully.
var ErrTargetUnknown = errors.New("target unknown")
ErrTargetUnknown is returned from a component.Linker when it does not know the appropriate target for a given aspect or interest.
var ErrTerminated = errors.New("component terminated")
ErrTerminated is the cause of L.Context() and L.GraceContext() cancellation due to a call to L.Terminate().
Functions ¶
func ExtractDoc ¶
ExtractDoc extracts a section of a package doc comment from the provided contents of a service component package's doc.go file.
A section is a portion of the comment between one heading and the next, using this form:
# Service NAME NAME: SUMMARY Full description...
where NAME matches the name argument, and SUMMARY is a brief verb-phrase that describes the service. The following lines, up until the next heading or the end of the comment, contain the full description. ExtractDoc returns the portion following the colon, which is the form expected by Descriptor.Doc.
Example:
# Service hello-world hello-world: consistently echoes hello-world to the console The hello-world service demonstrates a long-running service. Here is the complete description...
This notation allows a single doc comment to provide documentation for multiple services, each in its own section. The HTML anchors generated for each heading are predictable.
It returns an error if the content was not a valid Go source file containing a package doc comment with a heading of the required form.
This machinery enables the package documentation (typically accessible via the web at https://pkg.go.dev/) and the command documentation (typically printed to a terminal) to be derived from the same source and formatted appropriately.
func InjectLogger ¶
InjectLogger returns a new context based on the provided parent context, with the provided *slog.Logger associated with it.
func Logger ¶
Logger returns the *slog.Logger associated with the provided context. If none, it returns slog.Default.
func MergeTargets ¶
MergeTargets is a shorthand to concatenate targets (i.e., aspects and interests) into a single list, usually to serve as Descriptor.Interests.
func MustExtractDoc ¶
MustExtractDoc is like ExtractDoc but it panics on error.
to use, define a doc.go file in your package that contains a package comment with a heading for each service's component descriptor; like this:
// Package healthcheck defines a simple component of a subsystem. // // # Service healthcheck // // healthcheck: reports whether execution is alive. // // The healthcheck component reports a diagnostic for services // that run forever; to pass the diagnostics, try responding to // pings as fast as possible. package healthcheck import _ "embed" //go:embed doc.go var doc string
And declare your component as:
var Component = &component.Descriptor{
Name: "healthcheck",
Doc: loader.MustExtractDoc(doc, "healthcheck"),
...
}
func Run ¶
Run runs the provided procedure, passing it a new lifecycle.
The function blocks until the lifecycle has completed; i.e., until the main function has returned (or called L.Fatal), all its child lifecycles have completed, and all cleanup functions have been called.
Types ¶
type CleanupFunc ¶
type CleanupFunc func()
A CleanupFunc is a function that is called just before the lifecycle completes - either by returning from a Proc or by calling L.Fatal.
CleanupFuncs are called even if the lifecycle is terminated by calling Terminate or if the context is cancelled before the lifecycle completes - for this reason, cleanup functions must not rely on the context being valid.
See L.Cleanup, L.CleanupError and L.CleanupContext for more details.
type Descriptor ¶
type Descriptor struct {
// The Name of the component must be a valid Go identifier
// as it may appear in command-line flags, URLs, and so on.
Name string
// Doc is brief documentation for the component.
// The part before the first "\n\n" is the title
// (no capital or period, max ~60 letters).
// See ExtractDoc for details and examples.
Doc string
// Flag set defines any flags accepted by the component.
// The manner in which these flags are exposed to the user
// depends on the driver which runs the component.
Flags flag.FlagSet
// Bootstrap should run the component using the provided L.
// It returns an error if loading the component failed.
Bootstrap func(l *L, target Linker, options any) error
// OptionsType is the type of the "options" parameter passed to Bootstrap.
OptionsType reflect.Type
// Aspects define the pubsub topics that the component publishes to.
Aspects []string
// Interests define the pubsub topics that the component subscribes to.
Interests []string
}
A Descriptor describes a function of the system and its options.
type ForkOption ¶
type ForkOption func(*lifecycleOptions)
ForkOption is a function that configures a forked sub-lifecycle.
func WithForkCompletion ¶
func WithForkCompletion(done chan struct{}) ForkOption
WithForkCompletion closes the given channel to signal that the new lifecycle has completed its execution; that is, its root procedure, child lifecycles and its cleanup functions.
func WithForkName ¶
func WithForkName(name string) ForkOption
WithForkName sets the name of the new forked lifecycle. If no name is provided, the name of the program is used (i.e., os.Args[0]).
type L ¶
type L struct {
// contains filtered or unexported fields
}
L manages concurrent execution lifecycle and supports formatted logs.
A lifecycle ends when its Procedure returns or calls Fatal. This is the only way to exit a lifecycle. When called from another goroutine, Fatal will not be able to exit the lifecycle.
The other reporting methods, such as the variations of Log and Error, may be called simultaneously from multiple goroutines.
func (*L) Cleanup ¶
func (l *L) Cleanup(fn func())
Cleanup registers the given function to be called after the lifecycle has completed, in LIFO (stack) order.
Cleanup functions are called even if the lifecycle is terminated; however, the lifecycle context is cancelled at the time of termination, so cleanup should be aware of this.
Cleanup functions are called in the same goroutine as the lifecycle body.
func (*L) CleanupBackground ¶
CleanupBackground registers the given function to be called after the lifecycle has completed, like CleanupContext; However, a background context is passed to the function.
This helper is useful for calling cleanup functions that require a context, like http.Server.Shutdown(), but should not be cancelled when the lifecycle is terminated.
func (*L) CleanupContext ¶
CleanupContext registers the given function to be called after the lifecycle has completed, like CleanupError; However, the function is passed the context of the lifecycle.
This helper is useful for calling cleanup functions that require a context, like http.Server.Shutdown(). However, the context passed to the function may be cancelled by the time the function is called. Callers should be aware of this, or use CleanupBackground instead.
func (*L) CleanupError ¶
CleanupError registers the given function to be called after the lifecycle has completed, like Cleanup; However, if the function returns an error, it is logged using the Error() function.
This helper is useful for calling cleanup functions that return errors, such as io.Closer.Close().
func (*L) Continue ¶
Continue returns false if the lifecycle has been signalled to stop, otherwise it returns true indicating that the lifecycle should continue.
The following pattern is recommended:
for l.Continue() {
// do something, commonly with l.Context()
}
func (*L) Fatal ¶
Fatal behaves like Error except it terminates the lifecycle.
When called from goroutines other than the primary lifecycle goroutine, it can't terminate the lifecycle goroutine. Nonetheless, it will cancel the lifecycle context in order to signal to all goroutines that the lifecycle is terminating.
DO NOT call Fatal from goroutines other than those which started the lifecycle (i.e. the goroutine spawned by Run/L.Go/L.Fork/L.ForkE).
func (*L) Fork ¶
func (l *L) Fork(name string, procedure Procedure, opts ...ForkOption)
Fork derives a new lifecycle from the current lifecycle and executes the given Procedure in a new goroutine, passing this new lifecycle as its only argument.
This function returns without waiting for the new lifecycle completion and is safe for concurrent use.
The new lifecycle completes when the provided Procedure returns (and all the cleanup functions registered during its execution have returned as well).
The new lifecycle Context() is a child of the current lifecycle Context().
The new lifecycle Stopping() channel is closed when the current lifecycle Stopping() channel is closed.
Terminating the current lifecycle will terminate the new lifecycle as well.
This function panics if the current lifecycle is already stopping or if it has already completed.
func (*L) Go ¶
Go derives a new lifecycle from the current lifecycle and executes the given function in a new goroutine, passing this new lifecycle as its only argument.
This function returns without waiting for the new lifecycle completion and is safe for concurrent use.
The new lifecycle completes when the provided Proc returns (and all the cleanup functions registered during its execution have returned as well).
The new lifecycle Context() is a child of the current lifecycle Context().
The new lifecycle Stopping() channel is closed when the current lifecycle Stopping() channel is closed.
Terminating the current lifecycle will terminate the new lifecycle as well.
This function panics if the current lifecycle is already stopping or if it has already completed.
func (*L) GraceContext ¶
func (*L) Stop ¶
Stop returns true if the lifecycle has stopped gracefully within the timeout; otherwise, it returns false. This function is safe for concurrent use.
func (*L) Stopping ¶
func (l *L) Stopping() <-chan struct{}
Stopping returns a channel that is closed to signal the lifecycle procedure to stop gracefully; it is not closed when the lifecycle context expires.
The following pattern is recommended:
select {
case <-l.Stopping():
// gracefully stop
case <-l.Context().Done():
// handle timeout/abortion
}
type Linker ¶
type Linker interface {
// LinkAspect should return an event sink (a.k.a. target) for the given aspect
// based on its concomitant pubsub topic - as specified as part of a footprint.
//
// The event sink defines a point of egress from a component - that is,
// publication of notifications.
//
// The aspect must be one of those explicitly specified in Descriptor.Aspects,
// otherwise an error should be returned.
LinkAspect(ctx context.Context, aspect string) (target *pubsub.Topic, err error)
// LinkInterest should return an event source (a.k.a. target) for the given
// interest based on its concomitant pubsub topic - as specified as part of a
// footprint.
//
// The event source defines a point of ingress to a component - that is,
// subscription to notifications.
//
// The interest must be one of those explicitly specified in
// Descriptor.Interests, otherwise an error should be returned.
LinkInterest(ctx context.Context, interest string) (target *pubsub.Subscription, err error)
}
A Linker establishes a well-defined pathway that enabled two components to interact. That is, a linkage is implicitly formed when one component defines an aspect and another defines an interest, such that both the aspect and the interest reference that same target.
A target designates a point of data exchange; e.g., the name of a topic on a message broker.
Implementations of Linker should correlate aspects/interests with their respective data exchange targets based on different footprint specifications.
type Option ¶
type Option func(*lifecycleOptions)
Option is a function that configures a new lifecycle.
func OnComplete ¶
OnComplete returns an Option that adds a hook to be executed when the lifecycle is completed.
func OnStarted ¶
OnStarted returns an Option that adds a hook to be executed when the lifecycle is started.
func WithCompletion ¶
func WithCompletion(done chan struct{}) Option
WithCompletion closes the given channel to signal that the new lifecycle has completed its execution; that is, its root procedure, child lifecycles and its cleanup functions.
func WithContext ¶
WithContext sets the parent context of the new lifecycle. If no context is provided, a background context is used.
func WithForkSpanName ¶
WithForkSpanName overrides the name of the new forked lifecycle in the trace. If no name is provided, the name of the lifecycle is used.
func WithLogger ¶
WithLogger sets the logger used by the new lifecycle. If no logger is provided, a logger that discards all messages is used. The lifecycle shares this logger with all its children - started via L.Go().
TODO: using log.Llongfile causes the logger to print the wrong file because the call-site is in this package always.
func WithName ¶
WithName sets the name of the new lifecycle. If no name is provided, the name of the program is used (i.e., os.Args[0]).
func WithSpan ¶
WithSpan overrides the name of the new lifecycle in the trace. If no name is provided, the name of the lifecycle is used.
func WithStopper ¶
func WithStopper(stopper <-chan struct{}) Option
WithStopper monitors the given channel to signal that the new lifecycle should stop by receiving a value on the channel. If no stopper is provided, the lifecycle will not trigger a stop until manually requested via a call to L.Stop().
type Proc ¶
type Proc func(*L)
The Proc type is an adapter to allow the use of ordinary functions as component Procedure.
If f is a function with the appropriate signature, Proc(f) is a Procedure that calls f.
See notes on Procedure for more details about this function.
type ProcE ¶
The ProcE type is an adapter to allow the use of ordinary functions as component Procedure.
If f is a function with the appropriate signature, ProcE(f) is a Procedure that calls f and then calls Fatal if it returned a non-nil error.
See notes on Procedure for more details about this function.
type Procedure ¶
type Procedure interface {
Exec(*L)
}
The Procedure is the primary procedure of the component lifecycle.
During Exec authors should use the L parameter to interact with the current lifecycle:
- L.Go starts a new goroutine and returns immediately; the component does not complete until all goroutines started by L.Go have been completed.
- L.Cleanup (and variants) registers a function to be called when the component completes.
- L.Log and L.Error report logs and errors.
- L.Fatal terminates the component with the given error.
- L.Context returns the context associated with the component.
- L.Terminate terminates the component, causing L.Context to be canceled.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
examples
|
|
|
direct
command
This 'direct' example demonstrates how to use loader package to manually construct a loader.Footprint with resource allocations that link two components together - ping & pong.
|
This 'direct' example demonstrates how to use loader package to manually construct a loader.Footprint with resource allocations that link two components together - ping & pong. |
|
embed
command
This 'embed' example demonstrates how to use the loader package to manually construct a loader.Footprint with resource allocations that link three components together - ping, pong and probe.
|
This 'embed' example demonstrates how to use the loader package to manually construct a loader.Footprint with resource allocations that link three components together - ping, pong and probe. |
|
Package fileloader defines the main function for a component loader with only a constant footprint at runtime.
|
Package fileloader defines the main function for a component loader with only a constant footprint at runtime. |
|
Package healthprobe provides HTTP and gRPC health-check implementations that monitor the startup and liveness status of an application.
|
Package healthprobe provides HTTP and gRPC health-check implementations that monitor the startup and liveness status of an application. |
|
Package kafkalinker provides a Kafka pubsub implementation of the component.Linker interface.
|
Package kafkalinker provides a Kafka pubsub implementation of the component.Linker interface. |