Refactor into managed resources #14

Merged
antman merged 41 commits from resource-refactor into main 2026-08-19 08:38:44 +00:00
Owner

Motivation

We want to address the learnings from PR #12 — move away from the clumsy imperative implementation for installing applications and managing server state, and instead towards a managed resource model.

However, many insights occurred in the process of completing this refactor.

Solution

It turns out that a tree was not the correct data structure to model resources and their dependencies. Unfortunately, nodes can have more than one parent.

The extra flexibility that a tree structure would have provided turned out to be unnecessary. Currently, it seems incredibly likely that the direction of dependency will never differ from application to application. For example, a service will always depend upon the files, a caddy rule will always depend upon firewall rules, etc. There's no reason to expect a file will ever depend on a service being enabled first, or a firewall rule that depends upon a caddy rule.

Crucially, this allows us to keep the application manifest schema quite simple. If the tree data structure had been necessary, the application manifest schema would have had to represent the precise location of each and every resource in the dependency tree. We want to keep the application manifest scheme as simple as possible because this is an obvious place for third party developers to contribute. It's essentially the entry point into Kill the Cloud as a Platform where application developers can contribute applications for our servers.

This insight simplified implementing managers for each resource class. For example, a manager of file resources, a manager of packages, etc. Having a manager foreach resource class then made it possible to batch updates across an entire change set. They solved a number of problems updating packages in particular, since the removal of a package in one node of a tree didn't strictly mean that the package could now be removed because it may be required elsewhere. Having a package manager that only applied changes at the very end of a transaction meant that we could identify which packages to create, update, and delete by performing two asymmetrical difference operations and one intersection between the old resource set and the new resource set.

### Motivation We want to address the learnings from PR #12 — move away from the clumsy imperative implementation for installing applications and managing server state, and instead towards a managed resource model. However, many insights occurred in the process of completing this refactor. ### Solution It turns out that a tree was not the correct data structure to model resources and their dependencies. Unfortunately, nodes can have more than one parent. The extra flexibility that a tree structure would have provided turned out to be unnecessary. Currently, it seems incredibly likely that the direction of dependency will never differ from application to application. For example, a service will always depend upon the files, a caddy rule will always depend upon firewall rules, etc. There's no reason to expect a file will ever depend on a service being enabled first, or a firewall rule that depends upon a caddy rule. Crucially, this allows us to keep the application manifest schema quite simple. If the tree data structure had been necessary, the application manifest schema would have had to represent the precise location of each and every resource in the dependency tree. We want to keep the application manifest scheme as simple as possible because this is an obvious place for third party developers to contribute. It's essentially the entry point into **Kill the Cloud as a Platform** where application developers can contribute applications for our servers. This insight simplified implementing managers for each resource class. For example, a manager of file resources, a manager of packages, etc. Having a manager foreach resource class then made it possible to batch updates across an entire change set. They solved a number of problems updating packages in particular, since the removal of a package in one node of a tree didn't strictly mean that the package could now be removed because it may be required elsewhere. Having a package manager that only applied changes at the very end of a transaction meant that we could identify which packages to create, update, and delete by performing two asymmetrical difference operations and one intersection between the old resource set and the new resource set.
start sketching out resource implementation
All checks were successful
/ test-and-lint (push) Successful in 4m14s
8928a16ac6
implement more of file resource
All checks were successful
/ test-and-lint (push) Successful in 4m23s
01929a7532
implement file resource
All checks were successful
/ test-and-lint (push) Successful in 4m22s
c28a53535b
add tests
All checks were successful
/ test-and-lint (push) Successful in 4m27s
a667262ed4
Implement tests for file resource and root resource
All checks were successful
/ test-and-lint (push) Successful in 4m38s
95f1975fce
reorganize resource files
Some checks failed
/ test-and-lint (push) Failing after 32s
4dd07816fb
define a template resource to copy when implementing resources
Some checks failed
/ test-and-lint (push) Failing after 1m31s
4ceef270ee
before refactoring to use implementResource
All checks were successful
/ test-and-lint (push) Successful in 4m25s
68190c9505
Refactor to speedup resource implementation (#13)
Some checks failed
/ test-and-lint (push) Failing after 4m29s
d5c9b3dbf3
### Motivation

There are 12 resource implementations to write and maintain, that we know of so far. We want to minimize code duplicated amongst them as this will incur obvious maintenance burden, and it was already getting difficult to keep things consistent with just 2 implementations. The boilerplate code was getting quiet tedious when writing resource implementations, and it's likely that there will be more resource types to implement.

### Solution

I tried creating a higher order function that would take in another function that would be the resource creation function which the HOF would enhance. This turned out to be an inversion of what was required. Instead, the resource instance creation function (e.g. `defineFileResource`) calls a `getImplementers` function which takes a handful of arguments and returns a set of common properties and higher order functions that must wrap the CRUDD methods of a resource.

The downside of this approach is that anyone implementing a resource gets a little less guidance from the type system, but the upside is we don't have bidirectional data that they must flow between the higher order function and the concrete implementation, which gets very messy and confusing in a way outweighs the benefit of slightly better type hinting.

Co-authored-by: Anthony Manning-Franklin <anthony.manning.franklin@gmail.com>
Reviewed-on: #13
refactor hostname resource
Some checks failed
/ test-and-lint (push) Failing after 31s
3f6ebe944d
Re implement install app as resources
Some checks failed
/ test-and-lint (push) Failing after 5m16s
373d0b1fe0
argh
Some checks failed
/ test-and-lint (push) Failing after 4m47s
74ea9ee9d6
hmm
Some checks failed
/ test-and-lint (push) Failing after 4m48s
de55b5555b
fix file src path
Some checks failed
/ test-and-lint (push) Has been cancelled
8afd11ed30
implement file manager
Some checks failed
/ test-and-lint (push) Failing after 4m42s
fbc045d9b0
remove resource parents and children
Some checks failed
/ test-and-lint (push) Failing after 6s
dc7d344c56
remove diff and update methods
Some checks failed
/ test-and-lint (push) Failing after 6s
1533565a0a
make transactions global across resources
Some checks failed
/ test-and-lint (push) Failing after 6s
4e58746169
refactor to create managers programmatically
Some checks failed
/ test-and-lint (push) Failing after 5s
c6cdf463cc
define resource managers
Some checks failed
/ test-and-lint (push) Failing after 6s
b9b89f74c6
refactoring
Some checks failed
/ test-and-lint (push) Failing after 5s
1cd2e454c7
integrate new managed resource system into application
Some checks failed
/ test-and-lint (push) Failing after 7s
7783ccaf2d
fixed file tests
Some checks failed
/ test-and-lint (push) Failing after 7s
d71d5f5464
fix more stuff
Some checks failed
/ test-and-lint (push) Failing after 6s
afa04c3b0f
fix app boot sequence
Some checks failed
/ test-and-lint (push) Failing after 4m38s
580866fd9e
read hostname at applications startup
Some checks failed
/ test-and-lint (push) Failing after 4m41s
df96e00408
only run the problematic test
Some checks failed
/ test-and-lint (push) Has been cancelled
1350bf5dda
huh
Some checks failed
/ test-and-lint (push) Has been cancelled
2cb566e273
???
Some checks failed
/ test-and-lint (push) Has been cancelled
21b2e81773
more logs
Some checks failed
/ test-and-lint (push) Has been cancelled
9e2f1b9651
wtaf
Some checks failed
/ test-and-lint (push) Has been cancelled
4586f16a5c
duh
Some checks failed
/ test-and-lint (push) Failing after 4m39s
49c8b41845
@ -69,2 +69,4 @@
});
});
Deno.test('derivePatchedConfig does not change unrelated values', () => {
// TODO: Doesn't account for changing application data
Author
Owner

what does this mean?!

what does this mean?!
reinstate git clone
All checks were successful
/ test-and-lint (push) Successful in 4m16s
e11c5fad82
try installing all packages in a single command
Some checks failed
/ test-and-lint (push) Failing after 8s
c075c6282c
derp
All checks were successful
/ test-and-lint (push) Successful in 4m24s
b089eaca3c
fix
All checks were successful
/ test-and-lint (push) Successful in 3m54s
fbe9323e46
various cleanup
Some checks failed
/ test-and-lint (push) Failing after 5s
6cbe26a54a
vscode sucks
Some checks failed
/ test-and-lint (push) Failing after 4s
8e4ada8e2d
reeeeeeeeeeeeeeeeeeeeeee
Some checks failed
/ test-and-lint (push) Failing after 6s
eb63d89b27
aaaaaaaaaaaaaaaaa
All checks were successful
/ test-and-lint (push) Successful in 3m56s
5550ea36db
antman merged commit 5c7d0100e5 into main 2026-08-19 08:38:44 +00:00
antman deleted branch resource-refactor 2026-08-19 08:38:44 +00:00
antman referenced this pull request from a commit 2026-08-19 08:38:46 +00:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
kill-the-cloud/server-manager!14
No description provided.