1
0
mirror of https://github.com/ohwgiles/laminar.git synced 2024-10-27 20:34:20 +00:00
Commit Graph

34 Commits

Author SHA1 Message Date
Oliver Giles
41ddd8fe4f allow adding job to front of queue
laminarc now supports {queue,start,run} --next to place the job at the
front of the queue instead of at the end.

resolves #162
2022-01-22 15:18:43 +13:00
Oliver Giles
e581a0cf5d tests: add setNumExecutors method 2022-01-22 15:18:43 +13:00
Oliver Giles
4a6f99a203 tests: reinstantiate laminar for each unit
reinstantiate Laminar and Server classes and clean the temporary
LAMINAR_HOME directory for each unit test via SetUp and TearDown.
2022-01-22 15:18:43 +13:00
Oliver Giles
747ae3ada8 abort test: use "sleep inf" instead of "yes"
Sometimes yes can produce too much output too quickly and
overwhelm the host. It is not relevant to testing the abort
functionaliy.

resolves #143
2021-06-04 15:26:23 +12:00
Oliver Giles
746ab24676 move buildTimeDist chart to examples
too esoteric for the front page, converting it to an example
of querying laminar db directly and using gnuplot should provide
more value to those needing deeper insights into job behaviour.
2020-09-26 15:12:02 +12:00
Oliver Giles
d6be7f5079 fix minor compile warnings 2020-07-03 17:31:44 +12:00
Oliver Giles
2bf04d8157 allow setting a job description
adding DESCRIPTION=foo to $JOBNAME.conf will display "foo"
in the job overview page on the frontend

Resolves #97
2019-12-25 14:37:16 +02:00
Oliver Giles
3fde38c6b8 job leader process
Implement a separate process, the "leader", which runs all the
scripts for a job run, instead of directly from the main laminard
process. This makes for a cleaner process tree view, where the
owning job for a given script is clear; also the leader process
acts as a subreaper to clean up any wayward descendent processes.

Resolves #78.
2019-12-23 07:56:37 +02:00
Oliver Giles
0384fc9a0a
Replace nodes/tags with contexts
The nodes/tags system has not been particularly successful, it's not as
intuitive as it could be, and in an attempt to be a single feature to address
many use cases, ends up addressing none of them particularly well.

This commit replaces nodes and tags with contexts.

Each job may define which context(s) the job may be associated with.
Laminar will only pop the job off the waiting queue when it can be assigned
to a context. A context defines an integer number of executors, which
represents how many runs can be simultaneously assigned to it. A context
may provide extra environment variables.

Essentially, a context replaces a node, and tags are gone. You just assign
jobs to contexts directly, and you can use a glob expression. This should be
more intuitive.

For grouping jobs in the WebUI, a separate mechanism called "groups" is provided.
2019-12-13 10:42:22 +02:00
Oliver Giles
ba472711be refactor: remove run page json type hack
this hack tried to avoid sending unnecessary data to the frontend,
but it was more trouble than it's worth
2019-11-01 07:27:34 +02:00
Oliver Giles
39ca7e86cf replace websockets with sse and refactor
Large refactor that more closely aligns the codebase to the kj async
style, more clearly exposes an interface for functional testing and
removes cruft. There is a slight increase in coupling between the
Laminar and Http/Rpc classes, but this was always an issue, just until
now more obscured by the arbitrary pure virtual LaminarInterface class
(which has been removed in this change) and the previous lumping
together of all the async stuff in the Server class (which is now
more spread around the code according to function).

This change replaces the use of Websockets with Server Side Events
(SSE). They are simpler and more suitable for the publish-style messages
used by Laminar, and typically require less configuration of the
reverse proxy HTTP server.

Use of gmock is also removed, which eases testing in certain envs.

Resolves #90.
2019-10-05 20:06:35 +03:00
Oliver Giles
4a07e24da3 split server into http and rpc parts
this is initial preparation for a larger refactor
2019-09-27 20:50:46 +03:00
[[sroracle]]
1642899159 test: fix RunTest on systems with single file core utilities (#95)
On systems such as Alpine Linux (with busybox) and Adelie Linux (with
coreutils), the shell core utilities such as pwd, true, false, env, and
yes are all symlinks to a single binary. This single binary relies on
the name of the symlink to determine which command ("applet" in busybox
parlance) to execute. Therefore creating symlinks to these symlinks will
not work since the single binary will only see the top-level symlink and
thus think it is an invalid command.

Instead, generate executable shell scripts that exec into the desired
command. This also allows $PATH based resolution to occur instead of
hard-coding the command paths.

See also issue #94.
2019-05-24 21:27:24 +03:00
Oliver Giles
cec4721e52 resolves #79: serve logs over plain chunked http 2019-02-15 19:05:44 +02:00
Oliver Giles
7cee824cee resolves #67: laminarc list jobs
Implements the following laminarc commands:
- show-jobs
- show-running
- show-queued
- abort
2018-10-12 17:22:21 +03:00
Oliver Giles
08b3f25a22 Replace boost/filesystem with kj/filesystem
Lose the boost dependency since recent versions of capnproto's kj
also provide a nice filesystem library. Take the opportunity to
refactor the Run object to become more than POD and to encapsulate
some of the functionality that was done in the Laminar class

Part of #49 refactor
2018-09-28 12:43:40 +03:00
Oliver Giles
132d40e6a3 resolves #50: badge url
Implements serving of an SVG badge at the url /badge/JOB.svg which
prettily shows the job's current status
2018-09-10 14:51:43 +03:00
Oliver Giles
ab7be5a6c9 resolves #52: timeout prevents .after script
Mark .after scripts as executing even after a timeout abort
2018-09-09 12:15:23 +03:00
Oliver Giles
010af57ed4 resolves #45: new graphs
- regressions and recoveries: list of jobs whose run status changed,
  ordered first by currently failing jobs, secondly by count of jobs
  since the status change, descending for currently failing jobs and
  ascending for currently passing jobs
- low pass rates: list of the jobs with the worst pass rates calculated
  over all time
- run time changes: jobs with the largest changes in build time. This
  is calculated as the difference between the range and the standard
  deviation over the past 10 runs.
- average run time distribution: shows the number of jobs in the
  system divided into buckets based on their average runtime
2018-09-08 18:16:30 +03:00
Oliver Giles
a81492e5bc use kj's onChildExit mechanism
This reduces code and allows for more idiosyncratic use of Promises.
Requires latest capnproto git.

Part of #49 refactor
2018-08-03 14:36:24 +03:00
Oliver Giles
4ffc22c657 use promises to control job runs
This is a refactor that more cleanly uses the kj framework for handling
processes spawned by Runs. This obviates the workaround introduced back in
ff42dae7cc, and incidentally now requires c++14.

Part of #49 refactor
2018-07-20 17:40:56 +03:00
Oliver Giles
758b5f2e46 resolves #37: closed connection on large files
The old implementation slurped the whole artefact into memory, and
did not ensure it remained allocated beyond the first call to write().
The new implementation uses mmap and ensures the mapping lasts until
the file has been delivered
2018-07-06 13:18:04 +03:00
Oliver Giles
f1e4d10be3 resolves #36: queue/start/run 2018-05-12 17:56:56 +03:00
Oliver Giles
649caee297 resolves #30: job execution timeout
Add the ability to configure a timeout in seconds
after which a job run will be automatically aborted
2018-05-12 13:25:19 +03:00
Oliver Giles
083f136186 resolves #41: strange behaviour of limits
On a configuration change, update existing Nodes in Laminar's NodeMap
rather than replacing it with a new map.
2018-04-20 14:18:10 +03:00
Oliver Giles
216ecee7c5 resolves #34: watch for configuration changes
Reload the configuration if a change is detected
without requiring a server restart
2018-04-06 18:04:50 +03:00
Oliver Giles
9c256815e4 resolves #29: graceful shutdown
on SIGINT/SIGTERM:
1. stop accepting new connections
2. send SIGTERM to all child tasks
3. wait for processes to end
4. drop all websockets
2018-02-24 18:53:11 +02:00
Oliver Giles
30f2203a3b resolves #28: compensate for server/client clock skew 2018-02-03 16:52:46 +02:00
Oliver Giles
46efb07285 test: add LaminarTest stub 2018-02-03 16:47:41 +02:00
Oliver Giles
e7cba68879 tests: extract pointer to kj::AsyncIoStream
Fixes a crash that happens on g++ 6.3 under debian but not
g++ 7.2.1 under arch. Instead of trying to move kj::AsyncIoStream
between each promise handler, just extract and use the raw
pointer and let attach() clean up the Own<> at the end
2018-01-27 19:13:53 +02:00
Oliver Giles
3b267967cf resolves #31: exception thrown on bad file descriptor
Read or writes to bad file descriptors threw exceptions, just log
this instead. Also make sure that LaminarClients are removed from
the list of clients when a connection is aggressively dropped, and
add test
2018-01-27 13:11:40 +02:00
Oliver Giles
ade58a202a tests: env lives in /usr/bin 2018-01-27 13:06:06 +02:00
Oliver Giles
3bc60ec8f4 add basic server test 2018-01-27 12:59:39 +02:00
Oliver Giles
5ff3bbe2bb add basic tests for conf, database and run 2018-01-26 13:07:02 +02:00