Liking cljdoc? Tell your friends :D

monkey.ci.artifacts

Functionality for saving/restoring artifacts. This is similar to caches, but where caches are used between the same jobs in different builds, artifacts are used for different jobs in the same build. Artifacts can also be exposed to the outside world.

Functionality for saving/restoring artifacts.  This is similar to caches, but
where caches are used between the same jobs in different builds, artifacts are
used for different jobs in the same build.  Artifacts can also be exposed to
the outside world.
raw docstring

monkey.ci.blob

Blob storage functionality, used to store and restore large files or entire directories.

Blob storage functionality, used to store and restore large files
or entire directories.
raw docstring

monkey.ci.build.core

Core build script functionality. This is used by build scripts to create the configuration which is then executed by the configured runner. Which runner is configured or active depends on the configuration of the MonkeyCI application that executes the script.

This namespace provides low-level functions. For more user-friendly functionality, check out the monkey.ci.build.v2 namespace.

Core build script functionality.  This is used by build scripts to create
the configuration which is then executed by the configured runner.  Which
runner is configured or active depends on the configuration of the MonkeyCI
application that executes the script.

This namespace provides low-level functions.  For more user-friendly functionality,
check out the `monkey.ci.build.v2` namespace.
raw docstring

monkey.ci.build.shell

Low-level API functions to invoke shells from your build scripts.

Low-level API functions to invoke shells from your build scripts.
raw docstring

monkey.ci.build.v2

A group of functions written on top of the more low-level build functions, meant to improve writing build scripts. They should make the build scripts more intuitive, more readable. And, dare I say it, prettier?

The general intention is to provide functions for most purposes, without the user having to resort to using keywords and maps.

A group of functions written on top of the more low-level build functions, meant
to improve writing build scripts.  They should make the build scripts more intuitive,
more readable.  And, dare I say it, prettier?

The general intention is to provide functions for most purposes, without the user
having to resort to using keywords and maps.
raw docstring

monkey.ci.common.preds

Common predicate functions

Common predicate functions
raw docstring

monkey.ci.config

Configuration functionality. This reads the application configuration from various sources, like environment vars or command-line args. It provides some defaults and accessor functions. It mainly uses aero to reduce duplication in the configuration and merge multiple config files into one.

Configuration functionality.  This reads the application configuration from various
sources, like environment vars or command-line args.  It provides some defaults and
accessor functions.  It mainly uses aero to reduce duplication in the configuration
and merge multiple config files into one.
raw docstring

monkey.ci.console

Functionality for printing to the console, probably using ANSI coloring.

Functionality for printing to the console, probably using ANSI coloring.
raw docstring

monkey.ci.containers

Generic functionality for running containers

Generic functionality for running containers
raw docstring

monkey.ci.containers.podman

Functions for running containers using Podman. We don't use the api here, because it requires a socket, which is not always available. Instead, we invoke the podman command as a child process and communicate with it using the standard i/o streams.

Functions for running containers using Podman.  We don't use the api here, because
it requires a socket, which is not always available.  Instead, we invoke the podman
command as a child process and communicate with it using the standard i/o streams.
raw docstring

monkey.ci.containers.promtail

Functions for configuring promtail containers. This is not a container driver, but a utility namespace that is in turn used when pushing build logs to Loki.

Functions for configuring promtail containers.  This is not a container driver, but
a utility namespace that is in turn used when pushing build logs to Loki.
raw docstring

monkey.ci.core

Core namespace for the Monkey CI app. This contains the entrypoint which processes the configuration. This configuration determines whether the application runs as a server, execute a single script, which type of runner is enabled, etc...

Core namespace for the Monkey CI app.  This contains the entrypoint which
processes the configuration.  This configuration determines whether the
application runs as a server, execute a single script, which type of runner
is enabled, etc...
raw docstring

monkey.ci.cuid

Functions for working with cuids, which are like uuids but are a bit better to handle for humans.

Functions for working with cuids, which are like uuids but are a bit better
to handle for humans.
raw docstring

monkey.ci.dispatcher.core

The container dispatcher is responsible for picking pending container jobs or builds from a table, and assigning them to one of the available runners (either build or job). Since builds and container jobs both require containers, and as such compete for the same resources, they are merged into one by this dispatcher. Pending tasks (as we could call them) are saved in a table. This table is checked by the dispatcher whenever a job/pending, build/pending, job/end or build/end event is received. Depending on its strategy, it picks the next task (or tasks) to start and dispaches them to the registered runners, according to the available resources and requirements.

For example, some runners only support certain architectures, while others only have limited resources available. Some runners can have priority over others, probably to reduce costs (i.e. we would use the k8s runner before an oci runner).

The table is required to be able to pick from multiple waiting tasks, as opposed to an event queue, which will only allow to process them in sequence. Storing this in memory is not an option, since multiple replicas also mean multiple dispatchers. So this table is a way for them to sync up. It also keeps data over restarts.

Both builds and container jobs require infra resources to run, so it's logical that there is some sort of coordination. Various approaches are possible, but for now we have chosen the simplest approach: a single process is responsible for allocating the resources. It polls from the job and build topics as long as resources are available. If no more resources are available, it stops polling. Normally resources should become available again, and polling can resume. Any containers that are unassignable because they make requirements that can never be met (e.g. too many cpu's), are immediately marked as failed.

The available resources can be requested by the dispatcher, but for efficiency's sake it would be better that this information is kept locally, and updated by received events (build/start, job/start, build/end and job/end).

The container dispatcher is responsible for picking pending container jobs or builds
from a table, and assigning them to one of the available runners (either build or
job).  Since builds and container jobs both require containers, and as such compete
for the same resources, they are merged into one by this dispatcher.  Pending tasks
(as we could call them) are saved in a table.  This table is checked by the dispatcher
whenever a `job/pending`, `build/pending`, `job/end` or `build/end` event is received.
Depending on its strategy, it picks the next task (or tasks) to start and dispaches
them to the registered runners, according to the available resources and requirements.

For example, some runners only support certain architectures, while others only have
limited resources available.  Some runners can have priority over others, probably to
reduce costs (i.e. we would use the k8s runner before an oci runner).

The table is required to be able to pick from multiple waiting tasks, as opposed to
an event queue, which will only allow to process them in sequence.  Storing this
in memory is not an option, since multiple replicas also mean multiple dispatchers.
So this table is a way for them to sync up.  It also keeps data over restarts.

Both builds and container jobs require infra resources to run, so it's logical that
there is some sort of coordination.  Various approaches are possible, but for now we
have chosen the simplest approach: a single process is responsible for allocating the
resources.  It polls from the job and build topics as long as resources are available.
If no more resources are available, it stops polling.  Normally resources should become
available again, and polling can resume.  Any containers that are unassignable because
they make requirements that can never be met (e.g. too many cpu's), are immediately 
marked as failed.

The available resources can be requested by the dispatcher, but for efficiency's sake
it would be better that this information is kept locally, and updated by received
events (`build/start`, `job/start`, `build/end` and `job/end`).
raw docstring

monkey.ci.dispatcher.http

Http endpoints for the dispatcher. Mainly for monitoring.

Http endpoints for the dispatcher.  Mainly for monitoring.
raw docstring

monkey.ci.dispatcher.main

Main class for the dispatcher, used when running as a microservice

Main class for the dispatcher, used when running as a microservice
raw docstring

monkey.ci.dispatcher.spec

No vars found in this namespace.

monkey.ci.edn

Functionality for serializing objects to edn and deserializing them back.

Functionality for serializing objects to edn and deserializing them back.
raw docstring

monkey.ci.entities.core

Core functionality for database entities. Allows to store/retrieve basic entities.

Core functionality for database entities.  Allows to store/retrieve basic entities.
raw docstring

monkey.ci.entities.customer

Customer specific query functions

Customer specific query functions
raw docstring

monkey.ci.entities.repo

Repository specific query functions

Repository specific query functions
raw docstring

monkey.ci.entities.types

Special type treatments for sql

Special type treatments for sql
raw docstring

No vars found in this namespace.

monkey.ci.entities.webhook

Webhook related functionality

Webhook related functionality
raw docstring

monkey.ci.errors

Functions for working with errors and exceptions. This provides a uniform way to handle exceptions and convert them into a usable format for internal propagation and to give useful feedback to the user.

Functions for working with errors and exceptions.  This provides a uniform way
to handle exceptions and convert them into a usable format for internal 
propagation and to give useful feedback to the user.
raw docstring

monkey.ci.events.mailman.build-api

Mailman implementation that uses the build api for event posting and receiving. This is used by the build script.

Mailman implementation that uses the build api for event posting and
receiving.  This is used by the build script.
raw docstring

monkey.ci.events.mailman.jms

JMS-specific functionality for handling events. This has mainly to do with queue and topic configurations and listeners.

JMS-specific functionality for handling events.  This has mainly to do with
queue and topic configurations and listeners.
raw docstring

monkey.ci.extensions

Functionality for working with script extensions. Extensions are a way for third party libraries to add functionality to scripts, that is easy to activate and can also be used in yaml-type scripts. You could of course also add regular functions to invoke, but this is not easy to use, especially when using container jobs. Extensions do this by registering themselves under a specific namespaced keyword. If this key is found in job properties, the associated extension code is executed. Extensions can be executed before or after a job (or both).

Functionality for working with script extensions.  Extensions are a way
for third party libraries to add functionality to scripts, that is easy
to activate and can also be used in yaml-type scripts.  You could of 
course also add regular functions to invoke, but this is not easy to
use, especially when using container jobs.  Extensions do this by registering
themselves under a specific namespaced keyword.  If this key is found in
job properties, the associated extension code is executed.  Extensions can
be executed before or after a job (or both).
raw docstring

monkey.ci.git

Clone and checkout git repos. This is mostly a wrapper for clj-jgit

Clone and checkout git repos.  This is mostly a wrapper for `clj-jgit`
raw docstring

monkey.ci.k8s

Kubernetes functionality, used by build and container runners.

Kubernetes functionality, used by build and container runners.
raw docstring

monkey.ci.local.console

Event handlers for console reporting

Event handlers for console reporting
raw docstring

monkey.ci.local.events

Event routes for local build runners. When running a build locally via cli, these routes will be registered in mailman and will perform all necessary build steps.

Running a local build uses the same event flow as server-side builds. Depending on the configuration, routes may differ (e.g. different handlers or interceptors). A local build can either run in a child process, or it can run in a container. In any case, the container jobs are run in containers and action jobs are run by the script process.

Blobs (artifacts, caches) are always stored locally, but this can vary depending on configuration. If run in a container, a volume can be used.

The build controller, responsible for managing build params and events, is run in the main process. Builds connect to it using http, same as for server-side builds.

Event routes for local build runners.  When running a build locally via cli,
these routes will be registered in mailman and will perform all necessary
build steps.

Running a local build uses the same event flow as server-side builds.
Depending on the configuration, routes may differ (e.g. different handlers
or interceptors).  A local build can either run in a child process, or it
can run in a container.  In any case, the container jobs are run in containers
and action jobs are run by the script process.

Blobs (artifacts, caches) are always stored locally, but this can vary
depending on configuration.  If run in a container, a volume can be used.

The build controller, responsible for managing build params and events, is
run in the main process.  Builds connect to it using http, same as for 
server-side builds.
raw docstring

monkey.ci.local.runtime

Set up runtime for local builds

Set up runtime for local builds
raw docstring

monkey.ci.logging

Handles log configuration and how to process logs from a build script

Handles log configuration and how to process logs from a build script
raw docstring

monkey.ci.logging.loki

Logging implementation that sends to Loki. This is useful for the container implementation, where promtail is problematic because it never shuts down.

Logging implementation that sends to Loki.  This is useful for the container
implementation, where promtail is problematic because it never shuts down.
raw docstring

monkey.ci.process

Process execution functions. Executes build scripts in a separate process, using clojure cli tools.

Process execution functions.  Executes build scripts in a separate process,
using clojure cli tools.
raw docstring

monkey.ci.prometheus

Functionality to export metrics to prometheus format, or push them to a pushgateway. Originally we used micrometer-clj, but it's really old and incompatible with recent versions of the Prometheus libs. Since we're fixed on Prometheus (for now), it's not necessary to maintain all the other formats, so the micrometer layer was essentially ballast. This namespace accesses the Prometheus code directly.

Functionality to export metrics to prometheus format, or push them to a pushgateway.
Originally we used `micrometer-clj`, but it's really old and incompatible with recent
versions of the Prometheus libs.  Since we're fixed on Prometheus (for now), it's not
necessary to maintain all the other formats, so the micrometer layer was essentially
ballast.  This namespace accesses the Prometheus code directly.
raw docstring

monkey.ci.protocols

Contains all (or most of) the protocols used in the app. This is useful to avoid circular dependencies.

Contains all (or most of) the protocols used in the app.  This is useful
to avoid circular dependencies.
raw docstring

monkey.ci.reporting

Provides functions for reporting output. This can be logging, or printing to stdout, or formatting as json, etc...

Provides functions for reporting output.  This can be logging, or printing
to stdout, or formatting as json, etc...
raw docstring

monkey.ci.reporting.print

Reporter that prints to the console using coloring.

Reporter that prints to the console using coloring.
raw docstring

monkey.ci.runners.controller

Functions for running the application as a controller.

Functions for running the application as a controller.
raw docstring

monkey.ci.runners.k8s

No vars found in this namespace.

monkey.ci.runners.oci

Another implementation of a job runner that uses OCI container instances. This one uses mailman-style events instead of manifold. This should make it more robust and better suited for multiple replicas. Instead of waiting for a container instance to complete, we just register multiple event handlers that follow the flow.

Another implementation of a job runner that uses OCI container instances.
This one uses mailman-style events instead of manifold.  This should make
it more robust and better suited for multiple replicas.  Instead of waiting
for a container instance to complete, we just register multiple event 
handlers that follow the flow.
raw docstring

monkey.ci.runtime

The runtime can be considered the 'live configuration'. It is created from the configuration, and is passed on to the application modules. The runtime provides the information (often in the form of functions) needed by the modules to perform work. This allows us to change application behaviour depending on configuration, but also when testing.

This namespace also provides some utility functions for working with the runtime. This is more stable than reading properties from the runtime directly.

This namespace is being phased out in favor of passing specific information to components directly, since passing around too much information is an antipattern.

The runtime can be considered the 'live configuration'.  It is created
from the configuration, and is passed on to the application modules.  The
runtime provides the information (often in the form of functions) needed
by the modules to perform work.  This allows us to change application 
behaviour depending on configuration, but also when testing.

This namespace also provides some utility functions for working with the
runtime.  This is more stable than reading properties from the runtime 
directly.

This namespace is being phased out in favor of passing specific information
to components directly, since passing around too much information is an
antipattern.
raw docstring

monkey.ci.runtime.app

Functions for setting up a runtime for application (cli or server)

Functions for setting up a runtime for application (cli or server)
raw docstring

monkey.ci.script.config

Build script configuration functions, used by the process controller to create a valid configuration that can then be read by the build script runner.

Build script configuration functions, used by the process controller to 
create a valid configuration that can then be read by the build script runner.
raw docstring

monkey.ci.script.runtime

Functions for creating a runtime for build scripts

Functions for creating a runtime for build scripts
raw docstring

monkey.ci.sid

Functions for working with storage ids

Functions for working with storage ids
raw docstring

monkey.ci.sidecar.core

Sidecar specific functions

Sidecar specific functions
raw docstring

monkey.ci.sidecar.runtime

Functions for creating a runtime for a build-aware environment for the sidecar

Functions for creating a runtime for a build-aware environment for the sidecar
raw docstring

monkey.ci.spec.api-server

No vars found in this namespace.

monkey.ci.spec.app

No vars found in this namespace.

monkey.ci.spec.blob

No vars found in this namespace.

monkey.ci.spec.build

Spec definitions for build. The build object in the runtime and events that contain builds, scripts or jobs should conform to these specs.

Spec definitions for build.  The build object in the runtime and events that contain
builds, scripts or jobs should conform to these specs.
raw docstring

monkey.ci.spec.build-api

No vars found in this namespace.

monkey.ci.spec.config

Specs for app configurations

Specs for app configurations
raw docstring

No vars found in this namespace.

monkey.ci.spec.containers

Container related config and context specs

Container related config and context specs
raw docstring

No vars found in this namespace.

monkey.ci.spec.context

Job context spec

Job context spec
raw docstring

No vars found in this namespace.

monkey.ci.spec.db-entities

Spec for database entities. This can be useful to auto-generate database record entities for testing, but also to validate entities before persisting them.

Spec for database entities.  This can be useful to auto-generate database
record entities for testing, but also to validate entities before persisting 
them.
raw docstring

monkey.ci.spec.entities

Spec for application entities.

Spec for application entities.
raw docstring

monkey.ci.spec.events

Spec definitions for events

Spec definitions for events
raw docstring

monkey.ci.spec.extensions

No vars found in this namespace.

monkey.ci.spec.gen

Custom spec generators

Custom spec generators
raw docstring

monkey.ci.spec.job-context

No vars found in this namespace.

monkey.ci.spec.runner

No vars found in this namespace.

monkey.ci.spec.script

No vars found in this namespace.

monkey.ci.spec.sidecar

Specs for sidecar configuration

Specs for sidecar configuration
raw docstring

No vars found in this namespace.

monkey.ci.storage

Data storage functionality. Next to basic storage implementations, this ns also contains a lot of functions for working with storage entities. Many of these are overridden by implementation-specific functions, and so implementations here don't focus on efficiency. They are merely used in tests.

Data storage functionality.  Next to basic storage implementations, this ns also contains
a lot of functions for working with storage entities.  Many of these are overridden by
implementation-specific functions, and so implementations here don't focus on efficiency.
They are merely used in tests.
raw docstring

monkey.ci.storage.cached

Cached storage implementation. It wraps another storage and adds caching to it. This currently is a very naive implementation. It should be expanded with event processing, in case there are multiple replicas. Or we should replace it with a 'real' database.

Cached storage implementation.  It wraps another storage and adds caching to it.
This currently is a very naive implementation.  It should be expanded with event processing,
in case there are multiple replicas.  Or we should replace it with a 'real' database.
raw docstring

No vars found in this namespace.

monkey.ci.storage.file

File storage implementation. Useful for local or develop runs. It stores all information in local .edn files.

File storage implementation.  Useful for local or develop runs.  It stores all
information in local .edn files.
raw docstring

monkey.ci.storage.sql

Storage implementation that uses an SQL database for persistence. This namespace provides a layer on top of the entities namespace to perform the required queries whenever a document is saved or loaded.

Storage implementation that uses an SQL database for persistence.  This namespace provides
a layer on top of the entities namespace to perform the required queries whenever a 
document is saved or loaded.
raw docstring

monkey.ci.vault

Functions related to encryption/decryption of data using a vault

Functions related to encryption/decryption of data using a vault
raw docstring

monkey.ci.web.admin

API Route definitions for administrative purposes. These are not available in the general api and are meant to be used by system administrators or system processes only.

API Route definitions for administrative purposes.  These are not available
in the general api and are meant to be used by system administrators or
system processes only.
raw docstring

monkey.ci.web.api.params

Api functions for managing build parameters

Api functions for managing build parameters
raw docstring

monkey.ci.web.bitbucket

Bitbucket specific endpoints, mainly for authentication or push callbacks.

Bitbucket specific endpoints, mainly for authentication or push callbacks.
raw docstring

monkey.ci.web.middleware

Middleware for web requests. Could be that we move this to interceptors later.

Middleware for web requests.  Could be that we move this to interceptors later.
raw docstring

monkey.ci.web.oauth2

OAuth2 flow support handlers

OAuth2 flow support handlers
raw docstring

cljdoc is a website building & hosting documentation for Clojure/Script libraries

× close