Open for node port

.gitignore

@@ -10,4 +10,8 @@
 .env
 init-secrets.json
 .crd2pulumi-checksum
-/.clj-kondo/
+/.clj-kondo/
+kubeconfig.yaml
+/.cpcache/
+package-lock.json
+db.transit.json

README.md

@@ -3,119 +3,21 @@ My cluster configuration that serves to automate the deployment and handling of
 
 I'll try to include any pertinent documentation here in the tooling I use or the setup.
 
+#### This uses a very much alpha build of my pulumi-cljs library
 
 #### Upcoming
-Break this into three repos. IaC, the Pulumi CLJS library, and my dot files. We'll also be moving data from our old instances to the new IaC managed cluster. c:
-Current roadmap for that is breaking apart the Vault provider into its actual core components as it is currently an anti-pattern in the way it combines multiple provider functionalities through it. It relying on the config file even is a bit of an issue.
-Furthermore, we are unable to effectively destructure secrets in the execute function in the current design. However, since we'd want to change to remove the anti-pattern mentioned above, we'd ideally actually just reference secrets through the vault resource output from the given resource config's stack execution.
+Currently only want to expand on making the final service declarations functional.
+I'll be revising the library to better incorporate changes I'd like to see and I'd like to make the core cleaned up further.
+In the more immediate that'd be:
+- Simplifying the declarations
+- Pulling out the library functions
+- Cleaning up how re-used configs are appended to
 
-get-provider-outputs-config inside utils.providers.cljs currently runs under the expectation that shared stack already exists... which inherently is flawed on an initial run. Will need to revise a little. Similarly get-stack-refs works on the same flawed principle.
-Maybe we can move them to stack definitions (which currently exist in base.cljs). I think in an ideal design we could actually inherently scope the entire thing out. I'm inspired by how Guix allows system definitions to be written, and there isn't anything stopping a large block for a stack being like:
-```
-(def some-stack
-{:stack-registry
- [{:stack [:k8s:secret :k8s:chart]
-   :app-namespace "kube-system"
-   :app-name "hcloud-csi"
-   :vault-load-yaml false
-   :k8s:secret-opts {:metadata {:name "hcloud"
-                            :namespace "kube-system"}
-                 :stringData {:token  (-> cfg :hcloudToken)}}
-   :k8s:chart-opts {:fetchOpts {:repo "https://charts.hetzner.cloud"}
-                :values {:controller {:enabled false
-                                      :existingSecret {:name "hcloud-csi-secret"}
-                                      :node {:existingSecret {:name "hcloud-csi-secret"}}}}}}
-    {:stack [:vault:prepare :docker:image :k8s:secret :k8s:chart]
-   :app-namespace "caddy-system"
-   :app-name      "caddy-ingress-controller"
-   :k8s:image-port 8080
-   :k8s:vault-load-yaml false
-   :k8s:image-opts {:imageName '(str repo "/" app-name ":latest")}
-   :docker:image-opts {:registry {:server (-> cfg :public-image-registry-url)
-                                  :username (-> cfg :public-image-registry-username)
-                                  :password (-> cfg :public-image-registry-password)}
-                       :tags [(str (-> cfg :public-image-registry-url) "/" (-> cfg :public-image-registry-username) "/" "caddy")]
-                       :push true}
-   :k8s:chart-opts {:fetchOpts {:repo "https://caddyserver.github.io/ingress"}
-                    :values
-                    {:ingressController
-                     {:deployment {:kind "DaemonSet"}
-                      :daemonSet {:useHostPort true}
-                      :ports {:web {:hostPort 80}
-                              :websecure {:hostPort 443}}
-                      :service {:type "NodePort"
-                                :externalTrafficPolicy "Local"}
-                      :image {:repository 'repo
-                              :tag "latest"}
-                      :config {:email 'email}}}}}
-    ]
-    :stack-references { :init (new pulumi/StackReference "init") 
-                        :shared (new pulumi/StackReference "shared")}
-    :provider-configs {:harbor {:stack :shared
-                :outputs ["username" "password" "url"]}}
-    })
-```
-and that effectively defines an entire stack and is executable on (with the option to scope out to files to reduce the sheer verbosity in a single)
-In that regard, I think we've made decent headway in achieving a similar design and behavior where a config should provide reproducible results.
-Due to the nature of npm packages, it is a bit hard to *lock* to certain package versions as easily.
+Eventually the user will have more programmatic choice of execution too so as to have a program that ingests the library first and applies their stack defaults to it.
+The only limitation there would be that it does limit multi-cloud designs perhaps. Though for those, the unconsumed library would still be very much a practical option.
 
-DNS should be swapped with a Cloudflare provider instead and more appropriately allow EACH service to plainly define a DNS entry.
+To be more clear eventually rewriting the specs for defaults to match whatever stack configs given is likely the most optimal choice. Basically utilizing the homoiconic nature of Clojure and consuming the first program to generate the final one. Stubbing in the replacements as we walk through it.
 
-Local config loading or something should also be a provider, as obviously we would want to be able to pass through virtually anything to a service. That way they can be accessed later (this would replace the weird load-yaml that is a leftover from prior iterations)
-
-pulumi2crd should perhaps include some install instructions and some insight into usage
-Currently the script builds correctly, but since they are version dependant we might want to have some sort of version management for each of these components. That way they can be updated in a similar mechanism to a normal npm package. We can make CICD pipelines for them for this with some sort of cron scheduling. Emulating Renovate behavior (or we can see if Renovate can be useful here even).
-
-Those generated CRDs should not be baked into the provider utils but instead be treated as an expansion. This way it is neatly organized into official and extended functionality.
-
-Default values (like in K8) are opinionated. They do need to outline how to use a structure for example, but it should also be convenient to use any other resource like Nginx instead of Traefik or Azure instead of Cloudflare. A macro could be applied to them (preferably after their declaration, so their default state remains opinionated) to swap out which provider an individual chooses to use. It can be an added field in the core declarations for processing. Obvious goal for this is expansiveness. There should be clean, reusable defaults and everything should be easily modifiable and expandable.
-
-
-Resource declarations might benefit from being *able* to splinter when needed. Currently they are VERY MUCH locked to a singleton pattern. While we can "loop" over stuff inside a declaration it still only ever makes *one* resource.
-
-Currently, certificates relies upon a prior step existing and that in itself is a bit of an anti-pattern... So in the future our options NEED some way of informing the resolver and deployer that it has custom execution.
-```
-:k8s:certificates
-   {:constructor (.. cert-manager -v1 -Certificate)
-    :provider-key :k8s
-    :defaults-fn (fn [env]
-                   (p-> env :options :vault:prepare "stringData" .-domains
-                        #(vec
-                          (for [domain (js/JSON.parse %)] 
-                            (let [clean-name (clojure.string/replace domain #"\." "-")]
-                              {:_suffix clean-name
-                               :spec {:dnsNames [domain (str "*." domain)]
-                                      :secretName (str clean-name "-tls")}})))))}
-```
-The above is unideal. I think the best path forward for that is an override?  Considering that some might not use Vault.
-It might, instead, benefit from a high level user declaration of intent regarding the location of their secrets/settings. I mentioned above to have it resolve based on what providers utilized (within reason for support). That removes the inherent reliance, but it still does leave resolution in the default-fn in an unideal manner. It doesn't work to make top-level functions resolve on the outer layer as the Vault entry wouldn't exist yet. 
-If we do the user intent, we can at least change it to be a standard such as 
-```
-(p-> env :options :secrets .-domains #(function here))
-```
-I should add that this function would be in a more *plugin* since it isn't inherently a built-in for K8s. Same for Gateway.
-
-It wouldn't hurt to add some extension for developing these too. Increasing clarity on manner of declaration can not hurt.
-
-Should also revise default-fn to recursively call certificate and just allow the default-fn to unwind the values.
-
----
-It may be helpful to redesign the stack mechanism entirely so that resources and such are declared like:
-(def config
-    {:stack [
-        {:item-name 
-            {:options-in-here}} 
-        {:item-name-2 
-            {:options-in-here}}
-    ]})
-Where this provides much clearer association and each resource has its options readily available. As such you could declare duplicate keys in the same config. It would make resource associations much more explicit and cleaner written.
-It would require a decent amount of revision, so no rush on this.
-
-
-Mentioned above a bit, but eventually rewriting the specs for defaults to match whatever stack configs given is likely the most optimal choice. Basically utilizing the homoiconic nature of Clojure and consuming the first program to generate the final one. Stubbing in the replacements as we walk through it.
-
-
-Component spec really needs to be moved out of stack_processor as it is just such a large block of data that so better belongs w/ the providers themselves.
 
 ---
 
@@ -125,7 +27,6 @@ The long term goal is for this to be a mostly uninteractive, to completion set u
 More immediately, as we've closed in on a functional end-to-end alpha build and learned several choices we could've made to better design a next build, we'll actually use this to move our services off a single VPS w/ a docker compose and into a cluster fully generated by this with no setup or involvement on our part.
 
 
-
 ### Initial requirements
 #### Need to Revise as we swapped to using Pulumi Automation API so the entire process is automated
 
@@ -300,6 +201,25 @@ node -e 'console.log("admin:" + require("bcryptjs").hashSync("password", 10))'
 ```
 
 
+#### Restore Redis backup
+If the JuiceFS pods are crashing saying "Database not formatted."
+
+1. Download the backup Grab the latest JSON file from your pulumi-redis-backup bucket.
+2. Restore the Metadata You need to run the load command. You can do this from a temporary pod or any machine that can talk to the Redis service.
+Bash
+
+###### Spin up a temporary toolbox pod
+kubectl run -it --rm juicefs-restore --image=juicedata/mount:ce-v1.2.0 -- sh
+
+###### Inside the pod:
+1. Download your backup file (or copy/paste it if it's small)
+    (Assuming you copied the JSON content to a file named 'backup.json')
+2. Run the load command
+   Format: juicefs load redis://:PASS@HOST:PORT/DB backup.json
+juicefs load redis://:$(REDIS_PASS)@juicefs-redis.kube-system.svc.cluster.local:6379/1 backup.json
+3. Restart the JuiceFS Pods Once the load command finishes, the metadata is back. Delete the JuiceFS CSI pods (or the application pods using them) to force them to restart. They will connect, see the valid filesystem, and mount instantly.
+
+
 https://www.pulumi.com/registry/packages/docker-build/api-docs/image/
 https://www.pulumi.com/registry/packages/docker/api-docs/buildxbuilder/#create
 

deps.edn

@@ -0,0 +1,10 @@
+
+{:paths ["src/main"]
+ :deps {
+        ;;gigiaj/pulumicljs {:local/root "../pulumi-clojurescript"}
+        gigiaj/pulumicljs
+        {:git/url "https://github.com/GigiaJ/pulumi-clojurescript.git"
+         :git/sha "b7467d9da44a86a185e3bfe4307a7fe71add4134"
+         }
+        funcool/promesa {:mvn/version "11.0.678"}
+        }}

dummy-secrets.json

@@ -1,10 +1,142 @@
 {   "nextcloud": {
-        "password": "password",
-        "mariadb-root-password": "root-password",
-        "mariadb-password": "password",
-        "host": "nextcloud.example.com"
+        "password": "cute-lil-default-password",
+        "mariadb-root-password": "cute-lil-default-password",
+        "mariadb-password": "cute-lil-default-password",
+        "host": "nextcloud.enter-domain-here.com"
+    },
+    "mesite": {
+        "host": "enter-domain-here.com",
+        "registry-base": "harbor.enter-domain-here.com",
+        "registry-namespace": "registry-namespace"
+    },
+    "gitea": {
+        "host": "gitea.enter-domain-here.com",
+        "GITEA__service__DISABLE_REGISTRATION": false,
+        "GITEA__server__START_SSH_SERVER": true,
+        "GITEA__server__SSH_PORT": 22,
+        "GITEA__server__SSH_LISTEN_PORT": 2222,
+        "GITEA__server__SSH_DOMAIN": "gitea.enter-domain-here.com",
+        "repo": "docker.io/gitea"
+        
+    },
+    "act-runner": {
+        "GITEA_INSTANCE_URL": "https://gitea.enter-domain-here.com",
+        "GITEA_RUNNER_REGISTRATION_TOKEN": "REGISTRATION_TOKEN_HERE",
+        "GITEA_RUNNER_NAME": "Funky Sauce Runner",
+        "GITEA_RUNNER_LABELS": "my-host-runner:host,ubuntu-latest:host",
+        "GITEA_RUNNER_EPHEMERAL": 1,
+        "repo": "docker.io/gitea"
     },
     "dns": {
-        "service-sub-domain": "cloudflare-domain-zoneid"
+        "nextcloud": "ZONE_ID_HERE",
+        "productive": "ZONE_ID_HERE",
+        "harbor": "ZONE_ID_HERE",
+        "test-dnd": "ZONE_ID_HERE",
+        "gitea": "ZONE_ID_HERE",
+        "@": "ZONE_ID_HERE"
+    },
+    "superproductivity": {
+        "host": "productive.enter-domain-here.com",
+        "WEBDAV_BASE_URL": "https://nextcloud.enter-domain-here.com/remote.php/dav/files/your-name",
+        "WEBDAV_SYNC_FOLDER_PATH": "-/",
+        "SYNC_INTERVAL": "-15",
+        "IS_COMPRESSION_ENABLED": "-true",
+        "IS_ENCRYPTION_ENABLED": "-true"
+    },
+    "foundry": {
+        "host": "dnd.enter-domain-here.com",
+        "FOUNDRY_USERNAME": "username",
+        "FOUNDRY_PASSWORD": "password",
+        "registry-base": "harbor.enter-domain-here.com",
+        "registry-namespace": "registry-namespace"
+    },
+    "traefik": {
+        "domains": ["enter-domain-here.com", "enter-domain-here.com", "enter-domain-here.com", "enter-domain-here.com"],
+        "email": "your-name@enter-domain-here.com",
+        "repo": "https://traefik.github.io/charts",
+        "chart": "traefik"
+    },
+    "cert-manager": {
+        "domains": ["enter-domain-here.com", "enter-domain-here.com", "enter-domain-here.com", "enter-domain-here.com"],
+        "token": "DNS_PROVIDER_API_TOKEN",
+        "email": "your-name@enter-domain-here.com",
+        "secret-name": "api-token-secret"
+    },
+    "harbor": {
+        "host": "harbor.enter-domain-here.com",
+        "secret-key": "secret-key-here",
+        "username": "admin",
+        "password": "cute-lil-default-password",
+        "redis-username": "redis-user",
+        "redis-password": "cute-lil-default-password",
+        "REGISTRY_PASSWD": "cute-lil-default-password",
+        "REGISTRY_HTPASSWD": "cute-lil-default-password",
+        "htpasswd-string": "admin:cute-lil-encoded-default-password",
+        "harbor-secrets": "harbor-secrets-k8-resource-name",
+        "registry-secret": "16charsecrethere",
+        "jobservice-secret": "16charsecrethere",
+        "core-secret": "16charsecrethere",
+        "core-xrsf-key": "core-xrsf-key",
+        "core-token-key": "entire-rsa-key-here-with-line-breaks-stubbed-in-like-the-following\n",
+        "core-token-cert": "entire-cert-here-with-line-breaks-stubbed-in-like-the-following\n",
+        "admin-password": "cute-lil-default-password",
+        "db-password": "cute-lil-default-password",
+        "s3-access-key": "s3-access-key",
+        "s3-secret-key": "s3-secret-key",
+        "bucket": "your-s3-bucket-name",
+        "region": "us-east-1",
+        "repo": "https://helm.goharbor.io",
+        "region-endpoint": "http://wasabi-proxy.wasabi-proxy.svc.cluster.local"
+    },
+    "matrix-media-repo": {
+        "host": "",
+        "repo": "docker.io/turt2live"
+    },
+    "element-call": {
+        "host": "element-call.enter-domain-here.com",
+        "repo": "ghcr.io/element-hq"
+    },
+    "cinny": {
+        "host": "cinny.enter-domain-here.com",
+        "repo": "harbor.enter-domain-here.com/apps"
+    },
+    "element": {
+        "host": "element.enter-domain-here.com",
+        "repo": "vectorim/element-web:latest"
+    },
+    "mautrix-discord": {
+        "repo": "dock.mau.dev/mautrix"
+    },
+    "livekit-server": {
+        "repo": "docker.io/livekit"
+    },
+    "livekit-jwt": {
+        "host": "livekit.enter-domain-here.com",
+        "LIVEKit_SECRET": "LIVEKIT_SECRET_KEY_HERE",
+        "LIVEKIT_URL": "https://livekit.enter-domain-here.com/livekit/sfu",
+        "LIVEKIT_KEY":  "devkey",
+        "repo": "ghcr.io/element-hq"
+    },
+    "tuwunel": {
+        "repo": "ghcr.io/matrix-construct",
+        "host": "enter-domain-here.com",
+        "CONDUWUIT_SERVER_NAME": "enter-domain-here.com",
+        "CONDUWUIT_DATABASE_PATH": "/var/lib/conduwuit",
+        "CONDUWUIT_PORT": 6167,
+        "CONDUWUIT_MAX_REQUEST_SIZE": 20000000,
+        "CONDUWUIT_REGISTRATION_TOKEN": "REGISTRATION_TOKEN",
+        "CONDUWUIT_YES_I_AM_VERY_VERY_SURE_I_WANT_AN_OPEN_REGISTRATION_SERVER_PRONE_TO_ABUSE": false,
+        "CONDUWUIT_ALLOW_FEDERATION": true,
+        "CONDUWUIT_ALLOW_CHECK_FOR_UPDATES": true,
+        "CONDUWUIT_TRUSTED_SERVERS": ["matrix.org"],
+        "CONDUWUIT_TURN_URIS": ["turn:enter-domain-here.com?transport=udp", "turn:enter-domain-here.com?transport=tcp"],
+        "CONDUWUIT_ADDRESS": "0.0.0.0",
+        "CONDUWUIT_URL_PREVIEW_DOMAIN_EXPLICIT_ALLOWLIST": ["www.reddit.com", "www.youtube.com"],
+        "CONDUWUIT_ALLOW_LEGACY_MEDIA": true,
+        "CONDUWUIT_NEW_USER_DISPLAYNAME_SUFFIX": ""
+    },
+    "coturn": {
+        "repo": "docker.io/coturn",
+        "host": ""
     }
- }
+ }

package.json

@@ -30,8 +30,8 @@
     "@pulumi/pulumi": "^3.113.0",
     "@pulumi/vault": "^7.2.1",
     "@pulumiverse/harbor": "^3.10.21",
-    "@local/crds": "file:./generated/crds",
     "bcryptjs": "^3.0.2",
+    "pulumi-extra-crds": "1.0.13",
     "uuid": "^13.0.0"
   },
   "devDependencies": {

resources/backup-redis.yaml

@@ -0,0 +1,72 @@
+apiVersion: batch/v1
+kind: CronJob
+metadata:
+  name: juicefs-metadata-backup
+  namespace: kube-system
+spec:
+  schedule: "0 3 * * *"
+  successfulJobsHistoryLimit: 3
+  failedJobsHistoryLimit: 1
+  jobTemplate:
+    spec:
+      template:
+        spec:
+          restartPolicy: OnFailure
+          volumes:
+            - name: backup-dir
+              emptyDir: {}
+          
+          initContainers:
+            - name: dumper
+              image: juicedata/mount:ce-v1.2.0
+              command: ["/bin/sh", "-c"]
+              args:
+                - |
+                  echo "Starting JuiceFS Metadata Dump..."
+                  juicefs dump redis://:$(REDIS_PASS)@juicefs-redis.kube-system.svc.cluster.local:6379/1 > /backups/juicefs-meta.json
+                  if [ -s /backups/juicefs-meta.json ]; then
+                    echo "Dump successful (Size: $(du -h /backups/juicefs-meta.json | cut -f1))"
+                  else
+                    echo "Dump failed: File is empty"
+                    exit 1
+                  fi
+              volumeMounts:
+                - name: backup-dir
+                  mountPath: /backups
+              env:
+                - name: REDIS_PASS
+                  valueFrom:
+                    secretKeyRef:
+                      name: juicefs-redis-secrets
+                      key: password
+
+          containers:
+            - name: uploader
+              image: minio/mc:latest
+              command: ["/bin/sh", "-c"]
+              args:
+                - |
+                  echo "Configuring S3 Client..."
+                  mc alias set s3storage $S3_ENDPOINT $S3_ACCESS_KEY $S3_SECRET_KEY
+                  echo "Uploading backup..."
+                  FILENAME="juicefs-meta-$(date +%Y-%m-%d).json"
+                  mc cp /backups/juicefs-meta.json s3storage/$S3_BUCKET/backups/$FILENAME
+                  echo "Backup Complete!"
+              volumeMounts:
+                - name: backup-dir
+                  mountPath: /backups
+              env:
+                - name: S3_ENDPOINT
+                  value: "http://wasabi-proxy.wasabi-proxy.svc.cluster.local"
+                - name: S3_BUCKET
+                  value: "pulumi-redis-backup"
+                - name: S3_ACCESS_KEY
+                  valueFrom:
+                    secretKeyRef:
+                      name: juicefs-csi-secrets
+                      key: access-key
+                - name: S3_SECRET_KEY
+                  valueFrom:
+                    secretKeyRef:
+                      name: juicefs-csi-secrets
+                      key: secret-key

scripts/generate-crds.js

@@ -1,114 +0,0 @@
-const fs = require('fs');
-const path = require('path');
-const https = require('https');
-const crypto = require('crypto');
-const { execSync } = require('child_process');
-
-const PROJECT_ROOT = process.cwd(); 
-
-const OUTPUT_DIR = path.join(PROJECT_ROOT, 'generated/crds');
-const TEMP_DIR = path.join(PROJECT_ROOT, 'temp_crds');
-const CHECKSUM_FILE = path.join(PROJECT_ROOT, '.crd2pulumi-checksum');
-
-const GW_VERSION = 'v1.1.0';
-const GW_URL = `https://github.com/kubernetes-sigs/gateway-api/releases/download/${GW_VERSION}/experimental-install.yaml`;
-const GW_FILE = 'gateway-api.yaml';
-
-const CM_VERSION = 'v1.15.0';
-const CM_URL = `https://github.com/cert-manager/cert-manager/releases/download/${CM_VERSION}/cert-manager.crds.yaml`;
-const CM_FILE = 'cert-manager.yaml';
-
-const downloadFile = (url, filename) => {
-    return new Promise((resolve, reject) => {
-        const destPath = path.join(TEMP_DIR, filename);
-        const file = fs.createWriteStream(destPath);
-
-        const request = (uri) => {
-            https.get(uri, (response) => {
-                if (response.statusCode === 301 || response.statusCode === 302) {
-                    return request(response.headers.location);
-                }
-                if (response.statusCode !== 200) {
-                    reject(new Error(`Failed to download ${uri}: ${response.statusCode}`));
-                    return;
-                }
-                console.log(`Downloading ${filename}...`);
-                response.pipe(file);
-                file.on('finish', () => {
-                    file.close();
-                    resolve(destPath);
-                });
-            }).on('error', (err) => {
-                fs.unlink(destPath, () => {});
-                reject(err);
-            });
-        };
-        request(url);
-    });
-};
-
-const computeHash = (filePaths) => {
-    const hash = crypto.createHash('sha256');
-    filePaths.sort().forEach(fp => hash.update(fs.readFileSync(fp)));
-    return hash.digest('hex');
-};
-
-async function main() {
-    if (fs.existsSync(TEMP_DIR)) fs.rmSync(TEMP_DIR, { recursive: true, force: true });
-    fs.mkdirSync(TEMP_DIR);
-
-    try {
-        const gwPath = await downloadFile(GW_URL, GW_FILE);
-        const cmPath = await downloadFile(CM_URL, CM_FILE);
-        const allFiles = [gwPath, cmPath];
-        const newHash = computeHash(allFiles);
-
-        let oldHash = null;
-        if (fs.existsSync(CHECKSUM_FILE)) {
-            oldHash = fs.readFileSync(CHECKSUM_FILE, 'utf8').trim();
-        }
-
-        if (oldHash === newHash && fs.existsSync(OUTPUT_DIR)) {
-            console.log('CRDs unchanged. Skipping.');
-        } else {
-            console.log('Regenerating CRDs...');
-            if (fs.existsSync(OUTPUT_DIR)) fs.rmSync(OUTPUT_DIR, { recursive: true, force: true });
-            fs.mkdirSync(OUTPUT_DIR, { recursive: true });
-            execSync(`crd2pulumi --nodejsPath "${OUTPUT_DIR}" --force "${gwPath}" "${cmPath}"`, { stdio: 'inherit' });
-
-            const pkgPath = path.join(OUTPUT_DIR, 'package.json');
-            if (fs.existsSync(pkgPath)) {
-                const pkg = JSON.parse(fs.readFileSync(pkgPath, 'utf8'));
-                if (pkg.scripts) delete pkg.scripts;
-                fs.writeFileSync(pkgPath, JSON.stringify(pkg, null, 2));
-            }
-
-            const tsconfig = {
-                compilerOptions: {
-                    target: "es2020",
-                    module: "commonjs",
-                    moduleResolution: "node",
-                    declaration: true,
-                    skipLibCheck: true,
-                },
-                include: ["**/*.ts"],
-                exclude: ["node_modules"]
-            };
-            fs.writeFileSync(path.join(OUTPUT_DIR, 'tsconfig.json'), JSON.stringify(tsconfig, null, 2));
-            
-            try {
-                execSync('npx tsc', { cwd: OUTPUT_DIR, stdio: 'inherit' });
-            } catch (e) { console.warn("TSC warnings ignored."); }
-
-            fs.writeFileSync(CHECKSUM_FILE, newHash);
-            console.log(`Success!`);
-        }
-    } catch (error) {
-        console.error(error);
-        process.exit(1);
-    } finally {
-        if (fs.existsSync(TEMP_DIR)) fs.rmSync(TEMP_DIR, { recursive: true, force: true });
-    }
-}
-
-main();

shadow-cljs.edn

@@ -1,10 +1,9 @@
-{:source-paths ["./src/main"
-                "./generated/crds"]
- :dependencies [[funcool/promesa "11.0.678"]]
+{:dependencies []
+ :deps {:aliases []}
  :builds
  {:app {;;:target :node-library
-;;       :modules {:main {:entries [base]}}
-;;       :exports {:deployCore base/deploy-core}
+        ;;       :modules {:main {:entries [base]}}
+        ;;       :exports {:deployCore base/deploy-core}
         :target :node-script
         :output-to "./out/index.js"
         :compiler-options {:optimizations :simple}

src/main/core.cljs

@@ -1,20 +1,19 @@
 (ns core
   (:require
-   ["@pulumi/pulumi" :as pulumi]
    ["@pulumi/pulumi/automation" :as pulumi-auto]
    ["child_process" :as cp]
    [promesa.core :as p]
-   [base :as base]
    [configs :refer [cfg]]
-   [utils.execution.general :as general]
-   [utils.execution.providers :refer [execute]]
-   [service-registries :refer [base-resources-definition 
-                               initialize-resources-definition
-                               shared-resources-definition
-                               preparation-resources-definition
-                               deployment-resources-definition]]
+   [pulumicljs.execution.general :as general]
+   [pulumicljs.execution.providers :refer [execute]]
+   [stack-resource-definitions :refer [base-resources-definition 
+                                       initialize-resources-definition
+                                       shared-resources-definition
+                                       preparation-resources-definition
+                                       deployment-resources-definition
+                                       matrix-resources-definition]]
    )
-  (:require-macros [utils.execution.general :refer [p->]]))
+  (:require-macros [pulumicljs.execution.general :refer [p->]]))
 
 
 (defn define-stack [project-name stack-name work-dir program]
@@ -29,45 +28,53 @@
     "hetzner-k3s"
     "base"
     "/home/jaggar/dotfiles/iac"
-    (execute
-     base-resources-definition
-     #(#js {:kubeconfig (p-> % .-cluster "generic:execute" .-kubeconfig)}))))
+    #(execute
+      base-resources-definition
+      (fn [output] #js {:kubeconfig (p-> output .-cluster "generic:execute" .-kubeconfig)}))))
 
 (def init-stack
   (define-stack 
     "hetzner-k3s"
     "init"
     "/home/jaggar/dotfiles/iac"
-    (execute
+    #(execute
      initialize-resources-definition
-     #(#js {:vaultAddress (p-> % .-openbao "generic:execute" .-address)
-            :vaultToken (p-> % .-openbao "generic:execute" "root-token")}))))
+     (fn [output] #js {:vaultAddress (p-> output .-openbao "generic:execute" .-address)
+                       :vaultToken (p-> output .-openbao "generic:execute" "root-token")}))))
 
 (def shared-platform-stack
   (define-stack
     "hetzner-k3s"
     "shared"
     "/home/jaggar/dotfiles/iac"
-    (execute
+    #(execute
      shared-resources-definition
-     #(let [secrets (p-> % .-harbor "vault:prepare" "stringData")]
-        #js {:url (p-> secrets .-host (fn [x] (str "https://" x)))
-             :username (p-> secrets .-username)
-             :password (p-> secrets .-password)}))))
+     (fn [output] (let [secrets (p-> output .-harbor "vault:prepare" "stringData")]
+                    #js {
+                         :url (p-> secrets .-host (fn [x] (str "https://" x)))
+                         :username (p-> secrets .-username)
+                         :password (p-> secrets .-password)})))))
 
 (def prepare-deployment-stack
   (define-stack
     "hetzner-k3s"
     "prepare"
     "/home/jaggar/dotfiles/iac"
-    (execute preparation-resources-definition (fn [output] {}))))
+    #(execute preparation-resources-definition (fn [output] {}))))
 
 (def deployment-stack
   (define-stack
     "hetzner-k3s"
     "deployment"
     "/home/jaggar/dotfiles/iac"
-    (execute deployment-resources-definition (fn [output] {}))))
+    #(execute deployment-resources-definition (fn [output] {}))))
+
+(def matrix-stack
+  (define-stack
+    "hetzner-k3s"
+    "matrix"
+    "/home/jaggar/dotfiles/iac"
+    #(execute matrix-resources-definition (fn [output] {}))))
 
 
 (defn deploy-stack
@@ -78,7 +85,7 @@
    (p/let
     [stack (.createOrSelectStack pulumi-auto/LocalWorkspace stack-definition)
      _     (p/doseq [input inputs]
-             (.setConfig stack (:name input)  (clj->js (dissoc confinputig :name))))
+             (.setConfig stack (:name input)  (clj->js (dissoc input :name))))
      _     (.up stack #js {:onOutput println})
      outputs (.outputs stack)
      _     (p/delay post-delay)]
@@ -111,9 +118,10 @@
 
           shared-outputs (deploy-stack shared-platform-stack 
                                        (conj reused-configs {:name "hetzner-k3s:apiToken" :value (-> cfg :apiToken) :secret true})
-                                       1000)
+                                       1000) 
           prepare-outputs (deploy-stack prepare-deployment-stack reused-configs 3000)
           deployment-outputs (deploy-stack deployment-stack reused-configs 2000)
+          matrix-outputs (deploy-stack matrix-stack reused-configs 2000)
 
           _ (.kill port-forward)]
     "All stacks deployed and cleaned up successfully."))

src/main/infra/init.cljs

@@ -69,7 +69,10 @@
                                       {:direction "in" :protocol "udp" :port "51820" :sourceIps ["0.0.0.0/0" "::/0"]}
                                       {:direction "in" :protocol "tcp" :port "80"    :sourceIps ["0.0.0.0/0" "::/0"]}
                                       {:direction "in" :protocol "tcp" :port "443"    :sourceIps ["0.0.0.0/0" "::/0"]}
-                                      {:direction "in" :protocol "icmp"             :sourceIps ["0.0.0.0/0" "::/0"]}]}))
+                                      {:direction "in" :protocol "tcp" :port "30022"    :sourceIps ["0.0.0.0/0" "::/0"]}
+                                      {:direction "in" :protocol "icmp"             :sourceIps ["0.0.0.0/0" "::/0"]}
+                                      {:direction "in" :protocol "udp" :port "31000-31100" :sourceIps ["0.0.0.0/0" "::/0"]}
+                                      ]}))
 
         master     (hcloud/Server.
                     "k3s-master-de"

src/main/infra/openbao.cljs

@@ -272,9 +272,10 @@
   {:stack [:k8s:namespace :k8s:chart :generic:execute]
    :app-namespace "vault"
    :app-name "openbao"
+   :no-namespace true
    :exec-fn  execute-fn
    :vault-load-yaml false
-   :k8s:chart-opts {:fetchOpts {:repo "https://openbao.github.io/openbao-helm"}
+   :k8s:chart-opts {:repositoryOpts {:repo "https://openbao.github.io/openbao-helm"}
                 :transformations [(fn [props opts]
                                    (let [kind (:kind props)]
                                      (if (= kind "StatefulSet")

src/main/k8s/add_ons/cert_manager.cljs

@@ -5,11 +5,10 @@
    :app-namespace "cert-manager"
    :app-name "cert-manager"
    :is-prod? true
-   :k8s:chart-opts {:fetchOpts {:repo "https://charts.jetstack.io"}
+   :k8s:chart-opts {:repositoryOpts {:repo "https://charts.jetstack.io"}
                     :chart "cert-manager"
-                    :version "v1.15.0"
-                    :namespace "cert-manager"
-                    :values {:installCRDs true}}
+                    :version "v1.19.1"
+                    :namespace "cert-manager"}
    :k8s:secret-opts {:metadata {:name "api-token-secret"}
                      :stringData {:apiToken 'token}}
    :k8s:cluster-issuer-opts {:spec {:acme {:email 'email

src/main/k8s/add_ons/crd/cert_manager.cljs

@@ -0,0 +1,7 @@
+(ns k8s.add-ons.crd.cert-manager)
+
+(def config
+  {:stack [:k8s:config-file]
+   :app-name "cert-manager"
+   :version "v1.19.1"
+   :k8s:config-file-opts {:file '(str "https://github.com/cert-manager/cert-manager/releases/download/" version "/cert-manager.crds.yaml")}})

src/main/k8s/add_ons/crd/gateway_api.cljs

@@ -0,0 +1,7 @@
+(ns k8s.add-ons.crd.gateway-api)
+
+(def config
+  {:stack [:k8s:config-file]
+   :app-name "gateway-api"
+   :version "v1.4.0"
+   :k8s:config-file-opts {:file '(str "https://github.com/kubernetes-sigs/gateway-api/releases/download/" version "/experimental-install.yaml")}})

src/main/k8s/add_ons/crd/traefik.cljs

@@ -0,0 +1,10 @@
+(ns k8s.add-ons.crd.traefik)
+
+(def config
+  {:stack [:k8s:chart]
+   :app-name "traefik-crds"
+   :k8s:chart-opts {:repositoryOpts {:repo "https://traefik.github.io/charts"}
+                    :chart "traefik-crds"
+                    :version "v1.12.0"
+                    :namespace "traefik" 
+                    :values {:deleteOnUninstall true}}})

src/main/k8s/add_ons/csi_driver/extra/redis.cljs

@@ -0,0 +1,43 @@
+(ns k8s.add-ons.csi-driver.extra.redis)
+
+(def config
+  {:stack [:vault:prepare :k8s:secret :k8s:pvc :k8s:deployment :k8s:service]
+   :app-name "juicefs-redis"
+   :app-namespace "kube-system"
+   :no-namespace true
+
+   :k8s:pvc-opts
+   {:metadata {:name "juicefs-redis-data"
+               :namespace "kube-system"}
+    :spec {:accessModes ["ReadWriteOnce"]
+           :storageClassName "hcloud-volumes"
+           :resources {:requests {:storage "10Gi"}}}}
+
+   :k8s:deployment-opts
+   {:metadata {:name "juicefs-redis" :namespace "kube-system"}
+    :spec {:replicas 1
+           :selector {:matchLabels {:app "juicefs-redis"}}
+           :template {:metadata {:labels {:app "juicefs-redis"}}
+                      :spec {:volumes [{:name "juicefs-redis-data"
+                                        :persistentVolumeClaim
+                                        {:claimName "juicefs-redis-data"}}]
+                             :containers
+                             [{:name "juicefs-redis"
+                               :image "redis:7-alpine"
+                               :args ["--requirepass" "$(REDIS_PASS)"
+                                      "--maxmemory-policy" "noeviction"
+                                      "--appendonly" "yes"]
+                               :env [{:name "REDIS_PASS"
+                                      :valueFrom {:secretKeyRef {:name "juicefs-redis-secrets"
+                                                                 :key "password"}}}]
+                               :ports [{:containerPort 6379}]
+                               :volumeMounts [{:name "juicefs-redis-data"
+                                               :mountPath "/data"}]
+                               }]}}}}
+
+   :k8s:service-opts
+   {:metadata {:name "juicefs-redis" :namespace "kube-system"}
+    :spec {:type "ClusterIP"
+           :selector {:app "juicefs-redis"}
+           :ports [{:name 'app-name
+                    :port 6379 :targetPort 6379}]}}})

src/main/k8s/add_ons/csi_driver/hetzner.cljs

@@ -5,12 +5,13 @@
 (def config
   {:stack [:k8s:secret :k8s:chart]
    :app-namespace "kube-system"
+   :no-namespace true
    :app-name "hcloud-csi"
    :vault-load-yaml false
    :k8s:secret-opts {:metadata {:name "hcloud"
                             :namespace "kube-system"}
                  :stringData {:token  (-> cfg :hcloudToken)}}
-   :k8s:chart-opts {:fetchOpts {:repo "https://charts.hetzner.cloud"}
+   :k8s:chart-opts {:repositoryOpts {:repo "https://charts.hetzner.cloud"}
                 :values {:controller {:enabled false
                                       :existingSecret {:name "hcloud-csi-secret"}
                                       :node {:existingSecret {:name "hcloud-csi-secret"}}}}}})

src/main/k8s/add_ons/csi_driver/juicefs.cljs

@@ -0,0 +1,29 @@
+(ns k8s.add-ons.csi-driver.juicefs)
+
+(def config
+  {:stack [:vault:prepare :k8s:secret :k8s:chart :k8s:csi-driver :k8s:storage-class]
+   :app-namespace "kube-system"
+   :no-namespace true
+   :app-name "juicefs-csi"
+   :k8s:csi-driver-opts
+   {:metadata {:name "csi.juicefs.com"}
+    :spec {:attachRequired false
+           :podInfoOnMount true
+           :volumeLifecycleModes ["Persistent"]}}
+
+   :k8s:chart-opts
+   {:repositoryOpts {:repo "https://juicedata.github.io/charts/"}
+    :chart "juicefs-csi-driver"
+    :version "0.30.3"
+    :namespace "kube-system"
+    :values {:kubeletDir "/var/lib/kubelet"}}
+   :k8s:storage-class-opts
+   {:metadata {:name "juicefs-sc"}
+    :provisioner "csi.juicefs.com"
+    :parameters {"csi.storage.k8s.io/provisioner-secret-name" "juicefs-csi-secrets"
+                 "csi.storage.k8s.io/provisioner-secret-namespace" "kube-system"
+                 "csi.storage.k8s.io/node-publish-secret-name" "juicefs-csi-secrets"
+                 "csi.storage.k8s.io/node-publish-secret-namespace" "kube-system"
+                 "csi.storage.k8s.io/controller-expand-secret-name" "juicefs-csi-secrets"
+                 "csi.storage.k8s.io/controller-expand-secret-namespace" "kube-system"
+                 "pathPattern" "${.pvc.namespace}/${.pvc.name}"}}})

src/main/k8s/add_ons/csi_driver/wasabi.cljs

@@ -7,20 +7,10 @@
    :no-namespace true
    :app-name      "wasabi-csi"
    :k8s:chart-opts {:chart "csi-s3"
-                :fetchOpts {:repo "https://yandex-cloud.github.io/k8s-csi-s3/charts"}
+                :repositoryOpts {:repo "https://yandex-cloud.github.io/k8s-csi-s3/charts"}
                 :values {:controller {:enabled false
                                                         :existingSecret {:name "wasabi-csi-secrets"}
-                                                        :node {:existingSecret {:name "wasabi-csi-secrets"}}}}
-                                          
-                                          #_:storageClass #_{:create true
-                                                         :name "csi-s3-sc"
-                                                         :singleBucket "pulumi-harbor"
-                                                         :region "us-east-1"
-                                                         :accessKeyID "something"
-                                                         :secretAccessKey "something"
-                                                         ;;:bucket "pulumi-harbor"
-                                                         }}
+                                                        :node {:existingSecret {:name "wasabi-csi-secrets"}}}}}
    :k8s:secret-opts {:stringData {:accessKeyID (-> cfg :wasabiId)
                              :secretAccessKey (-> cfg :wasabiKey)
-                             :endpoint "http://wasabi-proxy.wasabi-proxy.svc.cluster.local"}}
-  :vault-load-yaml false})
+                             :endpoint "http://wasabi-proxy.wasabi-proxy.svc.cluster.local"}}})

src/main/k8s/add_ons/gateway/traefik.cljs

@@ -1,20 +1,42 @@
 (ns k8s.add-ons.gateway.traefik)
 
 (def config
-  {:stack [:vault:prepare [:k8s :secret :chart :gateway :certificates]]
+  {:stack [:vault:prepare [:k8s :secret :chart :gateway-class :gateway :certificates]]
    :app-namespace "traefik"
    :app-name "traefik"
    :is-prod? true
    :vault-load-yaml false
-   :k8s:chart-opts {:fetchOpts {:repo 'repo}
+   :k8s:chart-opts {:skipCrds true
+                    :repositoryOpts {:repo 'repo}
                     :chart 'chart
+                    :transformations  [(fn [args _opts] (let [kind (get-in args [:resource :kind])]
+                                       (if (= kind "CustomResourceDefinition")
+                                         nil
+                                         args)))]
                     :version "37.3.0"
                     :namespace "traefik"
                     :values {:providers {:kubernetesGateway {:enabled true}}
-                             :gatewayClass {:enabled true
-                                            :name "traefik"}}}
+                             :gatewayClass {:enabled false}
+                             :gateway {:enabled false}
+                             :ports {:web {:port 8000
+                                           :expose {:default true}
+                                           :exposedPort 80
+                                           :protocol "TCP"}
+
+                                     :websecure {:port 8443
+                                                 :expose {:default true}
+                                                 :exposedPort 443
+                                                 :protocol "TCP"
+                                                 :transport {:respondingTimeouts
+                                                             {:readTimeout "600s"
+                                                              :writeTimeout "600s"
+                                                              :idleTimeout "600s"}}}}}}
    :k8s:gateway-opts
    {:metadata {:name "main-gateway"
                :namespace "traefik"}
     :spec {:gatewayClassName "traefik"
-           :listeners '(make-listeners domains)}}})
+           :listeners '(make-listeners domains)}}
+   
+   :k8s:gateway-class-opts
+   {:spec {:controllerName "traefik.io/gateway-controller"}}
+   })

src/main/k8s/add_ons/image_registry/harbor.cljs

@@ -6,7 +6,7 @@
    :app-name "harbor"
    :image-port 80
    :vault-load-yaml false
-   :k8s:chart-opts {:fetchOpts {:repo "https://helm.goharbor.io"}
+   :k8s:chart-opts {:repositoryOpts {:repo "https://helm.goharbor.io"}
                     :values {:externalURL '(str "https://" host)
                              :expose {:type "route"
                                       :tls {:enabled false}

src/main/k8s/add_ons/ingress_controller/caddy.cljs

@@ -13,7 +13,7 @@
                                   :password (-> cfg :public-image-registry-password)}
                        :tags [(str (-> cfg :public-image-registry-url) "/" (-> cfg :public-image-registry-username) "/" "caddy")]
                        :push true}
-   :k8s:chart-opts {:fetchOpts {:repo "https://caddyserver.github.io/ingress"}
+   :k8s:chart-opts {:repositoryOpts {:repo "https://caddyserver.github.io/ingress"}
                     :values
                     {:ingressController
                      {:deployment {:kind "DaemonSet"}

src/main/k8s/add_ons/proxy.cljs

@@ -54,6 +54,7 @@
    :k8s:service-opts
    {:spec
     {:ports
-     [{:port 80
+     [{:name 'app-name
+       :port 80
        :targetPort 80}]}}})
 

src/main/k8s/add_ons/secret_replicator.cljs

@@ -6,4 +6,4 @@
    :no-namespace true
    :app-namespace "kube-system"
    :app-name      "kubernetes-replicator"
-   :k8s:chart-opts {:fetchOpts {:repo "https://helm.mittwald.de"}}})
+   :k8s:chart-opts {:repositoryOpts {:repo "https://helm.mittwald.de"}}})

src/main/k8s/services/foundryvtt/service.cljs

@@ -1,7 +1,8 @@
 (ns k8s.services.foundryvtt.service)
 
+;; Need to automate license unlock
 (def config
-  {:stack [:vault:prepare :harbor:robot-account :docker:image [:k8s :deployment :service :httproute]]
+  {:stack [:vault:prepare :harbor:robot-account :docker:image [:k8s :pvc :deployment :service :httproute]]
    :image-port     30000
    :app-namespace "generic"
    :app-name      "foundry"
@@ -14,11 +15,23 @@
                        :tags ['(str registry-base "/" registry-namespace "/" app-name)]
                        :push true}
    :k8s:deployment-opts {:spec {:template {:spec {:imagePullSecrets [{:name "harbor-creds-secrets"}]
-                                                  :containers [{:name 'app-name :image '(str registry-base "/" registry-namespace "/" app-name ":latest")}]}}}}
+                                                  :volumes [{:name "data-vol"
+                                                            :persistentVolumeClaim {:claimName "vtt-assets"}}]
+                                                  :containers [{:name 'app-name :image '(str registry-base "/" registry-namespace "/" app-name ":latest")
+                                                                :volumeMounts [{:name "data-vol"
+                                                                                :mountPath "/root/.local/share"
+                                                                                :mountPropagation "HostToContainer"}]
+                                                                }]}}}}
    :harbor:robot-account-opts {:name 'app-name
                                :permissions [{:kind "project"
                                               :namespace 'registry-namespace
                                               :access [{:action "pull" :resource "repository"}
                                                        {:action "push" :resource "repository"}
                                                        {:action "list" :resource "repository"}]}]}
+   :k8s:pvc-opts
+   {:metadata {:name "vtt-assets"
+               :namespace "generic"}
+    :spec {:storageClassName "juicefs-sc"
+           :accessModes ["ReadWriteMany"] 
+           :resources {:requests {:storage "10Gi"}}}}
    :k8s:httproute-opts {:spec {::hostnames ['host]}}})

src/main/k8s/services/foundryvtt/service_2.cljs

@@ -0,0 +1,21 @@
+(ns k8s.services.foundryvtt.service-2)
+
+(def config
+  {:stack [:vault:prepare [:k8s :pvc :deployment :service :httproute]]
+   :image-port     30000
+   :app-namespace "generic"
+   :app-name      "girls-foundry" 
+   :k8s:deployment-opts {:spec {:template {:spec {:imagePullSecrets [{:name "harbor-creds-secrets"}]
+                                                  :volumes [{:name "data-vol"
+                                                             :persistentVolumeClaim {:claimName "girls-vtt-assets"}}]
+                                                  :containers [{:name 'app-name :image '(str registry-base "/" registry-namespace "/" "foundry" ":latest")
+                                                                :volumeMounts [{:name "data-vol"
+                                                                                :mountPath "/root/.local/share"
+                                                                                :mountPropagation "HostToContainer"}]}]}}}} 
+   :k8s:pvc-opts
+   {:metadata {:name "girls-vtt-assets"
+               :namespace "generic"}
+    :spec {:storageClassName "juicefs-sc"
+           :accessModes ["ReadWriteMany"]
+           :resources {:requests {:storage "10Gi"}}}}
+   :k8s:httproute-opts {:spec {::hostnames ['host]}}})

src/main/k8s/services/gitea/service.cljs

@@ -1,25 +1,51 @@
 (ns k8s.services.gitea.service)
 
 (def config
-  {:stack [:vault:prepare :k8s:deployment :k8s:service :k8s:httproute]
-   :image-port    3000
+  {:stack [:vault:prepare :k8s:pvc :k8s:deployment :k8s:service :k8s:httproute :k8s:tcproute]
    :app-namespace "generic"
    :app-name      "gitea"
-   :k8s:deployment-opts {:spec {:template {:spec {:initContainers [
-                                                               {:name "init-permissions"
-                                                                :image "busybox:latest"
-                                                                :command ["sh" "-c" "chown -R 1000:1000 /var/lib/gitea && chown -R 1000:1000 /etc/gitea"]
-                                                                :volumeMounts [{:name "gitea-data" :mountPath "/var/lib/gitea"}
-                                                                               {:name "gitea-config" :mountPath "/etc/gitea"}]
-                                                                :securityContext {:runAsUser 0 :runAsGroup 0}}
-                                                               ]
-                                              :containers [{:name 'app-name :image '(str repo "/" app-name ":latest-rootless")
-                                                            :volumeMounts [{:name "gitea-data" :mountPath "/var/lib/gitea"}
-                                                                           {:name "gitea-config" :mountPath "/etc/gitea"}
-                                                                           {:name "timezone" :mountPath "/etc/timezone" :readOnly true}
-                                                                           {:name "localtime" :mountPath "/etc/localtime" :readOnly true}]}]
-                                              :volumes [{:name "gitea-data" :hostPath {:path "/opt/gitea/data" :type "DirectoryOrCreate"}}
-                                                        {:name "gitea-config" :hostPath {:path "/opt/gitea/config" :type "DirectoryOrCreate"}}
-                                                        {:name "timezone" :hostPath {:path "/etc/timezone" :type "File"}}
-                                                        {:name "localtime" :hostPath {:path "/etc/localtime" :type "File"}}]}}}}
-   :k8s:httproute-opts {:spec {::hostnames ['host]}}})
+   :image-port 3000
+   :k8s:pvc-opts
+   {:metadata {:name "gitea-state"
+               :namespace "generic"}
+    :spec {:storageClassName "juicefs-sc"
+           :accessModes ["ReadWriteMany"]
+           :resources {:requests {:storage "1Ti"}}}}
+
+   :k8s:deployment-opts
+   {:spec
+    {:template
+     {:metadata {:annotations {"backup.velero.io/backup-volumes" "gitea-state"}}
+      :spec
+      {:containers
+       [{:name 'app-name
+         :image '(str repo "/" app-name ":latest-rootless")
+         :command ["/usr/local/bin/gitea"]
+         :args ["web"
+                "-c" "/var/lib/gitea/custom/conf/app.ini"]
+         :env [{:name "TZ" :value "America/Chicago"}]
+         :envFrom [{:secretRef {:name "gitea-secrets"}}]
+         :ports [{:name "ssh"  :containerPort 2222}]
+
+         :volumeMounts [{:name "gitea-state" :mountPath "/var/lib/gitea"}]}]
+
+       :volumes
+       [{:name "gitea-state"
+         :persistentVolumeClaim {:claimName "gitea-state"}}]}}}}
+
+   :k8s:service-opts
+   {:spec
+    {:type "NodePort"
+     :selector {:app "gitea"}
+     :ports [{:name 'app-name :port 3000 :targetPort 3000}
+
+             {:name "ssh"
+              :port 22
+              :targetPort 2222
+              :nodePort 30022}]}}
+   :k8s:httproute-opts {:spec {::hostnames ['host]
+                               :rules [{:matches [{:path {:type "PathPrefix"
+                                                          :value "/"}}]
+                                        :backendRefs [{:name 'app-name
+                                                       :port 3000}]}]}}})
+

src/main/k8s/services/homeassistant/service.cljs

@@ -0,0 +1,34 @@
+(ns k8s.services.homeassistant.service)
+
+(def config
+  {:stack [:vault:prepare :k8s:pvc :k8s:deployment :k8s:service :k8s:httproute]
+   :image-port    8123
+   :app-namespace "home"
+   :app-name      "homeassistant"
+
+   :k8s:pvc-opts
+   {"ha-config" {:storageClass "hcloud-volumes"
+                 :accessModes ["ReadWriteOnce"]
+                 :storage "10Gi"}}
+
+   :k8s:deployment-opts
+   {:spec
+    {:template
+     {:spec
+      {:containers
+       [{:name 'app-name
+         :image '(str repo  "/home-assistant:stable")
+         :env [{:name "TZ" :value "America/Chicago"}]
+         :volumeMounts [{:name "config" :mountPath "/config"}]}]
+       :volumes
+       [{:name "config" :persistentVolumeClaim {:claimName "ha-config"}}]}}}}
+
+   :k8s:service-opts
+   {:spec {:selector {:app 'app-name}
+           :ports [{:name 'app-name :port 8123 :targetPort 8123}]}}
+
+   :k8s:httproute-opts
+   {:spec
+    {:hostnames ['host]
+     :rules [{:matches [{:path {:type "PathPrefix" :value "/"}}]
+              :backendRefs [{:name 'app-name :port 8123}]}]}}})

src/main/k8s/services/matrix/cinny/service.cljs

@@ -1,12 +1,29 @@
 (ns k8s.services.matrix.cinny.service)
 
 (def config
-  {:stack [:vault-secrets :docker-image :deployment :service :ingress]
-   :image-port     80
+  {:stack [:vault:prepare
+           :harbor:robot-account
+           :docker:image
+           [:k8s :deployment :service :httproute]]
    :app-namespace "matrix"
    :app-name      "cinny"
-   :image-opts {:build {:args {:FOUNDRY_USERNAME 'FOUNDRY_USERNAME
-                               :FOUNDRY_PASSWORD 'FOUNDRY_PASSWORD}}
-                :imageName '(str repo "/" app-name ":latest")}
-   :deployment-opts {:spec {:template {:spec {:imagePullSecrets [{:name "harbor-creds-secrets"}]
-                                              :containers [{:name 'app-name :image '(str repo "/" app-name ":latest")}]}}}}})
+   :docker:image-opts {:context {:location "https://github.com/GigiaJ/cinny.git"}
+                       :imageName '(str registry-base "/" registry-namespace "/" app-name ":latest")
+                       :registry {:server '(str registry-base "/" registry-namespace)
+                                  :username '(-> :harbor:robot-account .-name)
+                                  :password '(-> :harbor:robot-account .-secret)}
+                       :tags ['(str registry-base "/" registry-namespace "/" app-name)]
+                       :push true}
+
+   :harbor:robot-account-opts {:name 'app-name
+                               :permissions [{:kind "project"
+                                              :namespace 'registry-namespace
+                                              :access [{:action "pull" :resource "repository"}
+                                                       {:action "push" :resource "repository"}
+                                                       {:action "list" :resource "repository"}]}]}
+
+   :k8s:deployment-opts {:spec {:template {:spec {:imagePullSecrets [{:name "harbor-creds-secrets"}]
+                                                  :containers [{:name 'app-name
+                                                                :image '(str registry-base "/" registry-namespace "/" app-name ":latest")
+                                                                :ports [{:containerPort 80}]}]}}}}
+   :k8s:httproute-opts {:spec {::hostnames ['host]}}})

src/main/k8s/services/matrix/database/service.cljs

@@ -1,13 +0,0 @@
-(ns k8s.services.matrix.database.service)
-;;     env_file:
-;;      - .env
-;;    volumes:
- ;;     - ${PWD}/db-data/:/var/lib/postgresql/data/
-
-
-(def config
-  {:stack [:deployment :service :ingress]
-   :image-port     80
-   :app-namespace "matrix"
-   :app-name      "postgres"
-   :deployment-opts {:spec {:template {:spec {:containers [{:name 'app-name :image '(str repo "/" 'app-name ":latest")}]}}}}})

src/main/k8s/services/matrix/element/service.cljs

@@ -1,14 +1,69 @@
-;;    volumes:
-;;      - ./personal/matrix/element-config.json:/app/config.json
-;;    environment:
-;;      ELEMENT_WEB_PORT: 3030
-
 (ns k8s.services.matrix.element.service)
 
 (def config
-  {:stack [:vault-secrets :docker-image :deployment :service :ingress]
-   :image-port     80
+  {:stack [:vault:prepare [:k8s :config-map :deployment :service :httproute]]
+   :image-port    80
    :app-namespace "matrix"
-   :app-name      "element"
-   :deployment-opts {:spec {:template {:spec {:imagePullSecrets [{:name "harbor-creds-secrets"}]
-                                              :containers [{:name 'app-name :image '(str repo "/" app-name ":latest")}]}}}}})
+   :app-name      "element-web"
+   :k8s:config-map-opts {:data {"config.json"
+                                '(stringify
+                                  {:default_server_name homeserver
+                                   :default_server_config
+                                   {:m.homeserver
+                                    {:base_url (str "https://" homeserver)}
+                                    :m.identity_server
+                                    {:base_url identity-server}}
+
+                                   :brand brand-name
+
+                                   :integrations_ui_url "https://scalar.vector.im/"
+                                   :integrations_rest_url "https://scalar.vector.im/api"
+                                   :integrations_widgets_urls
+                                   ["https://scalar.vector.im/_matrix/integrations/v1"
+                                    "https://scalar.vector.im/api"
+                                    "https://scalar-staging.vector.im/_matrix/integrations/v1"
+                                    "https://scalar-staging.vector.im/api"
+                                    "https://scalar-staging.riot.im/scalar/api"]
+
+                                   :bug_report_endpoint_url "https://element.io/bugreports/submit"
+                                   :uisi_autorageshake_app "element-auto-uisi"
+                                   :show_labs_settings true
+                                   :room_directory
+                                   {:servers [homeserver]}
+                                   :enable_presence_by_hs_url
+                                   {"https://matrix.org" false
+                                    "https://matrix-client.matrix.org" false}
+                                   :terms_and_conditions_links
+                                   [{:url (str "https://" homeserver "/privacy")
+                                     :text "Privacy Policy"}
+                                    {:url (str "https://" homeserver "/cookie-policy")
+                                     :text "Cookie Policy"}]
+                                   :sentry
+                                   {:dsn "https://029a0eb289f942508ae0fb17935bd8c5@sentry.matrix.org/6"
+                                    :environment "develop"}
+                                   :posthog
+                                   {:project_api_key "phc_Jzsm6DTm6V2705zeU5dcNvQDlonOR68XvX2sh1sEOHO"
+                                    :api_host (str "https://posthog." homeserver)}
+                                   :privacy_policy_url (str "https://" homeserver "/cookie-policy")
+                                   :features
+                                   {:threadsActivityCentre true
+                                    :feature_video_rooms true
+                                    :feature_group_calls true
+                                    :feature_element_call_video_rooms true}
+                                   :setting_defaults
+                                   {:RustCrypto.staged_rollout_percent 100
+                                    :Registration.mobileRegistrationHelper true}
+                                   :element_call
+                                   {:url (str "https://livekit." homeserver)}
+                                   :map_style_url "https://api.maptiler.com/maps/streets/style.json?key=fU3vlMsMn4Jb6dnEIFsx"})}}
+   :k8s:deployment-opts {:spec
+                         {:template
+                          {:spec
+                           {:volumes [{:name "config-vol"
+                                       :configMap {:name 'app-name}}]
+                            :containers [{:name 'app-name :image '(str repo "/" app-name ":latest")
+                                          :env [{:name "ELEMENT_WEB_PORT" :value "80"}]
+                                          :volumeMounts [{:name "config-vol"
+                                                          :mountPath "/app/config.json"
+                                                          :subPath "config.json"}]}]}}}}
+   :k8s:httproute-opts {:spec {::hostnames ['host]}}})

src/main/k8s/services/matrix/element_call/livekit_jwt/service.cljs

@@ -0,0 +1,22 @@
+(ns k8s.services.matrix.element-call.livekit-jwt.service)
+
+(def config
+  {:stack [:vault:prepare [:k8s :deployment :service :httproute]]
+   :image-port     8080
+   :app-namespace "matrix"
+   :app-name      "livekit-jwt"
+   :k8s:deployment-opts {:spec
+                         {:template
+                          {:spec
+                           {:containers [{:name 'app-name :image '(str repo "/" "lk-jwt-service" ":latest")
+                                          :env [{:name "LIVEKIT_KEY"    :value 'key-name}
+                                                {:name "LIVEKIT_SECRET" :value 'dev-key}
+                                                {:name "LIVEKIT_JWT_PORT" :value "8080"}
+                                                {:name "LIVEKIT_URL"    :value 'livekit-url}]}]}}}}
+   :k8s:httproute-opts
+   {:spec
+    {:hostnames ['host]
+     :rules [{:matches [{:path {:type "PathPrefix" :value "/livekit/jwt"}}]
+              :backendRefs [{:name 'app-name :port 80}]}
+             {:matches [{:path {:type "PathPrefix" :value "/sfu/get"}}]
+              :backendRefs [{:name 'app-name :port 80}]}]}}})

src/main/k8s/services/matrix/element_call/livekit_server/service.cljs

@@ -0,0 +1,76 @@
+(ns k8s.services.matrix.element-call.livekit-server.service)
+
+(defn generate-all-ports [tcp-port start-udp end-udp]
+  (concat
+   [{:name "http"
+     :port tcp-port
+     :targetPort tcp-port
+     :containerPort tcp-port
+     :protocol "TCP"}]
+
+   (map (fn [p]
+          {:name (str "udp-" p)
+           :port p
+           :targetPort p
+           :nodePort p
+           :containerPort p
+           :protocol "UDP"})
+        (range start-udp (inc end-udp)))))
+
+(def all-ports (generate-all-ports 7880 31000 31100))
+
+
+(def config
+  {:stack [:vault:prepare [:k8s :config-map :deployment :service :httproute]]
+   :image-port    nil
+   :app-namespace "matrix"
+   :app-name      "livekit-server"
+
+   :k8s:config-map-opts
+   {:metadata {:name "livekit-config"}
+    :data {"livekit.yaml"
+           '(stringify
+             {:port 7880
+              :bind_addresses ["0.0.0.0"]
+              :rtc {:tcp_port 7881
+                    :port_range_start 31000
+                    :port_range_end 31100
+                    :use_external_ip true} ;; Required for Hetzner Public IP discovery
+
+              :logging {:level "debug"}
+              :turn {:enabled false
+                     :udp_port 443
+                     :tls_port 5349}
+
+              :keys {:devkey dev-key}})}}
+   
+   :k8s:deployment-opts
+   {:spec
+    {:template
+     {:spec
+      {:volumes [{:name "config-vol" :configMap {:name "livekit-config"}}]
+       :containers [{:name 'app-name
+                     :image '(str repo "/" app-name ":latest")
+                     :command ["/livekit-server"]
+                     :args ["--config" "/etc/livekit.yaml"]
+                     :ports (map #(select-keys % [:name :containerPort :protocol])
+                                 all-ports)
+                     :volumeMounts [{:name "config-vol"
+                                     :mountPath "/etc/livekit.yaml"
+                                     :subPath "livekit.yaml"}]}]}}}}
+
+   :k8s:service-opts
+   {:spec {:type "NodePort"
+           :selector {:app 'app-name}
+           :ports (map #(select-keys % [:name :port :targetPort :nodePort :protocol])
+                       all-ports)}}
+
+   :k8s:httproute-opts
+   {:spec
+    {:hostnames ['host]
+     :rules [{:matches [{:path {:type "PathPrefix" :value "/livekit/sfu"}}]
+              :filters [{:type "URLRewrite"
+                         :urlRewrite {:path {:type "ReplacePrefixMatch"
+                                             :replacePrefixMatch "/"}}}]
+
+              :backendRefs [{:name 'app-name :port 7880}]}]}}})

src/main/k8s/services/matrix/element_call/service.cljs

@@ -1,18 +1,26 @@
 (ns k8s.services.matrix.element-call.service)
 
-;;    volumes:
-;;      - ./personal/matrix/elementcall/config.json:/app/config.json
-
 (def config
-  {:stack [:vault-secrets :deployment :service :ingress]
+  {:stack [:vault:prepare [:k8s :config-map :deployment :service :httproute]]
    :image-port    80
    :app-namespace "matrix"
    :app-name      "element-call"
-   :deployment-opts {:spec {:template {:spec {:containers [{:name 'app-name :image '(str repo "/" app-name ":sha-1702b15")
-                                                            :volumeMounts [{:name "data" :mountPath "/data"}]}]
-                                              :initContainers [{:name "init-permissions"
-                                                                :image "busybox:latest"
-                                                                :command ["sh" "-c" "chown -R 1000:1000 /data"]
-                                                                :volumeMounts [{:name "data" :mountPath "/data"}]
-                                                                :securityContext {:runAsUser 0 :runAsGroup 0}}]
-                                              :volumes [{:name "data" :hostPath {:path "/opt/mmr/data" :type "DirectoryOrCreate"}}]}}}}})
+   :k8s:config-map-opts {:data {"config.json"
+                                '(stringify
+                                  {:default_server_config
+                                   {:m.homeserver
+                                    {:base_url (str "https://" homeserver)
+                                     :server_name homeserver}}
+                                   :features
+                                   {:feature_use_device_session_member_events true}
+                                   :ssla "https://static.element.io/legal/element-software-and-services-license-agreement-uk-1.pdf"})}}
+   :k8s:deployment-opts {:spec
+                         {:template
+                          {:spec
+                           {:volumes [{:name "config-vol"
+                                       :configMap {:name 'app-name}}]
+                            :containers [{:name 'app-name :image '(str repo "/" app-name ":latest")
+                                          :volumeMounts [{:name "config-vol"
+                                                          :mountPath "/app/config.json"
+                                                          :subPath "config.json"}]}]}}}}
+   :k8s:httproute-opts {:spec {::hostnames ['host]}}})

src/main/k8s/services/matrix/home_server/service.cljs

@@ -1,10 +1,53 @@
-;;  homeserver:
-;;    volumes:
-;;        - db:/var/lib/conduwuit
- 
- (def config
-  {:stack [:vault-secrets :docker-image :deployment :service :ingress]
-   :image-port     80
+(ns k8s.services.matrix.home-server.service)
+
+(def config
+  {:stack [:vault:prepare [:k8s :pvc :deployment :service :httproute]]
    :app-namespace "matrix"
    :app-name      "tuwunel"
-   :deployment-opts {:spec {:template {:spec {:containers [{:name 'app-name :image '(str repo "/" 'app-name ":latest")}]}}}}})
+   
+   
+   :k8s:pvc-opts
+   {:metadata {:name "conduwuit-db"
+               :namespace "matrix"}
+    :spec {:storageClassName "hcloud-volumes"
+           :accessModes ["ReadWriteOnce"]
+           :resources {:requests {:storage "50Gi"}}}}
+
+   :k8s:deployment-opts
+   {:spec
+    {:strategy {:type "Recreate"}
+     :template
+     {:metadata {:annotations {"backup.velero.io/backup-volumes" "db"}}
+      :spec
+      {:containers
+       [{:name 'app-name
+         :image '(str repo "/tuwunel:latest")
+         :envFrom [{:secretRef {:name '(str app-name "-secrets")}}]
+         :ports [{:containerPort 'port}]
+         :volumeMounts [{:name "db" :mountPath "/var/lib/conduwuit"}
+                        #_{:name "discord-reg"
+                         :mountPath "/etc/conduwuit/discord-registration.yaml"
+                         :subPath "registration.yaml"}]}]
+
+       :volumes
+       [{:name "db" :persistentVolumeClaim {:claimName "conduwuit-db"}}
+        #_{:name "discord-reg" :configMap {:name "discord-bridge-config"}}
+        ]}}}}
+
+   :k8s:service-opts
+   {:spec {:ports [{:name 'app-name :port 'port :targetPort 'port}]}}
+
+   :k8s:httproute-opts
+   {:spec
+    {:hostnames ['host]
+     :rules [{:matches [{:path {:type "PathPrefix" :value "/_matrix/media"}}]
+              :backendRefs [{:name "matrix-media-repo" :port 80}]}
+
+             {:matches [{:path {:type "PathPrefix" :value "/_matrix/client/v1/media"}}]
+              :backendRefs [{:name "matrix-media-repo" :port 80}]}
+
+             {:matches [{:path {:type "PathPrefix" :value "/.well-known/matrix"}}]
+              :backendRefs [{:name "matrix-well-known" :port 80}]}
+
+             {:matches [{:path {:type "PathPrefix" :value "/"}}]
+              :backendRefs [{:name 'app-name :port 'port}]}]}}})

src/main/k8s/services/matrix/home_server/well_known/service.cljs

@@ -0,0 +1,27 @@
+(ns k8s.services.matrix.home-server.well-known.service)
+
+(def config
+  {:stack [:vault:prepare :k8s:config-map :k8s:deployment :k8s:service]
+   :app-namespace "matrix"
+   :app-name      "matrix-well-known"
+
+   :k8s:config-map-opts
+   {:metadata {:name "well-known-json"}
+    :data {"server" "{\"m.server\": \"hampter.quest:443\"}"
+           "client" '(stringify
+                      {:m.homeserver {:base_url (str "https://" homeserver)}
+                       :org.matrix.msc4143.rtc_foci [{:type "livekit"
+                                                      :livekit_service_url livekit-url}]})}}
+   :k8s:deployment-opts
+   {:spec
+    {:template
+     {:spec
+      {:containers
+       [{:name 'app-name
+         :ports [{:containerPort 80}]
+         :image '(str repo "/nginx:alpine")
+         :volumeMounts [{:name "config" :mountPath "/usr/share/nginx/html/.well-known/matrix"}]}]
+       :volumes [{:name "config" :configMap {:name "well-known-json"}}]}}}}
+
+   :k8s:service-opts
+   {:spec {:ports [{:name 'app-name :port 80 :targetPort 80}]}}})

src/main/k8s/services/matrix/livekit_jwt/service.cljs

@@ -1,8 +0,0 @@
-(ns k8s.services.matrix.livekit-jwt.service)
-
-(def config
-  {:stack [:vault-secrets :docker-image :deployment :service :ingress]
-   :image-port     80
-   :app-namespace "matrix"
-   :app-name      "livekit-jwt"
-   :deployment-opts {:spec {:template {:spec {:containers [{:name 'app-name :image '(str repo "/" lk-jwt-service ":0.2.3")}]}}}}})

src/main/k8s/services/matrix/livekit_server/service.cljs

@@ -1,16 +0,0 @@
-
-
-;;  livekit:
-;;    command: --config /etc/livekit.yaml
-;;      - ./personal/matrix/elementcall/livekit.yaml:/etc/livekit.yaml
-  ;;  ports:
-   ;;   - 50100-50200:50100-50200/udp
-
-(ns k8s.services.matrix.livekit-server.service)
-
-(def config
-  {:stack [:vault-secrets :docker-image :deployment :service :ingress]
-   :image-port     80
-   :app-namespace "matrix"
-   :app-name      "livekit-server"
-   :deployment-opts {:spec {:template {:spec {:containers [{:name 'app-name :image '(str repo "/" 'app-name ":latest")}]}}}}})

src/main/k8s/services/matrix/mautrix_discord/database/service.cljs

@@ -0,0 +1,37 @@
+(ns k8s.services.matrix.mautrix-discord.database.service)
+
+(def config
+  {:stack [:vault:prepare :k8s:pvc :k8s:deployment :k8s:service]
+   :app-namespace "matrix"
+   :app-name      "mautrix-discord-db"
+
+   :k8s:pvc-opts
+   {:metadata {:name "mautrix-discord-pg-data"
+               :namespace "matrix"}
+    :spec {:storageClassName "hcloud-volumes"
+           :accessModes ["ReadWriteOnce"]
+           :resources {:requests {:storage "10Gi"}}}}
+
+   :k8s:deployment-opts
+   {:spec
+    {:template
+     {:metadata
+      {:annotations
+       {"backup.velero.io/backup-volumes" "db"}}
+      :spec 
+      {:containers
+       [{:name 'app-name
+         :image "postgres:14-alpine"
+         :ports [{:containerPort 5432}]
+         :env [{:name "PGDATA" :value "/var/lib/postgresql/data/pgdata"}
+               {:name "POSTGRES_USER" :value 'username}
+               {:name "POSTGRES_PASSWORD" :value 'password}
+               {:name "POSTGRES_DB" :value 'db-name}]
+         :volumeMounts [{:name "db" :mountPath "/var/lib/postgresql/data"}]}]
+
+       :volumes
+       [{:name "db" :persistentVolumeClaim {:claimName "mautrix-discord-pg-data"}}]}}}}
+
+   :k8s:service-opts
+   {:spec {:selector {:app 'app-name}
+           :ports [{:name 'app-name :port 5432 :targetPort 5432}]}}})

src/main/k8s/services/matrix/mautrix_discord/service.cljs

@@ -1,10 +1,78 @@
-;;     - ./personal/matrix/discord/data:/data
-
 (ns k8s.services.matrix.mautrix-discord.service)
 
 (def config
-  {:stack [:vault-secrets :docker-image :deployment :service :ingress]
-   :image-port     80
+  {:stack [:vault:prepare :k8s:pvc :k8s:config-map :k8s:deployment :k8s:service]
    :app-namespace "matrix"
    :app-name      "mautrix-discord"
-   :deployment-opts {:spec {:template {:spec {:containers [{:name 'app-name :image '(str repo "/" "discord" ":4927a73ce7411f3970803d35c22f0c8c96dc2d7e-amd64")}]}}}}})
+
+   :k8s:config-map-opts
+   {:metadata {:name "discord-bridge-config"}
+    :data {"config.yaml"       '(stringify
+                                 {:homeserver {:address (str "https://" homeserver)
+                                               :domain homeserver}
+                                  :appservice {:port port
+                                               :address (str "http://mautrix-discord:" port)
+                                               :hostname "0.0.0.0"
+                                               :database {:type "postgres"
+                                                          :uri db-login-url
+                                                          :max_open_conns 20
+                                                          :max_idle_cons 2}
+                                               :id "discord"
+                                               :as_token as-token
+                                               :hs_token hs-token
+                                               :ephemeral_events true
+                                               :bot {:username "discordbot"
+                                                     :displayname "Discord bridge bot"}}
+                                  :bridge {:permissions (parse permissions)
+                                           ;;:login_shared_secret_map (parse login-shared-secret-map)
+                                           ;;:double_puppet_server_map (parse double-puppet-server-map)
+                                           :use_discord_cdn_upload true
+                                           :command_prefix "!discord"
+                                           :encryption {:allow false
+                                                        :default false}}})
+           "registration.yaml" '(stringify {:id "discord"
+                                            :url (str "http://mautrix-discord:" port)
+                                            :as_token as-token
+                                            :hs_token hs-token
+                                            :sender_localpart sender-localpart
+                                            :rate_limited false
+                                            :namespaces {:users [{:regex user1-regex
+                                                                  :exclusive true}
+                                                                 {:regex user2-regex
+                                                                  :exclusive true}]}
+                                            :de.sorunome.msc2409.push_ephemeral true
+                                            :push_ephemeral true})}}
+
+   
+   :k8s:pvc-opts
+   {:metadata {:name "discord-bridge-data"
+               :namespace "matrix"}
+    :spec {:storageClassName "juicefs-sc"
+           :accessModes ["ReadWriteMany"]
+           :resources {:requests {:storage "1Gi"}}}}
+   :k8s:deployment-opts
+   {:spec
+    {:template
+     {:spec
+      {:initContainers
+       [{:name "config-loader"
+         :image "busybox:latest"
+         :command ["sh" "-c" "cp -f /config_source/* /data/"]
+         :volumeMounts [{:name "data" :mountPath "/data"}
+                        {:name "config" :mountPath "/config_source"}]}]
+       :containers
+       [{:name 'app-name
+         :image '(str repo "/discord:latest")
+         :args ["/usr/bin/mautrix-discord" "-c" "/data/config.yaml" "-r" "/data/registration.yaml"]
+         :ports [{:containerPort 'port}]
+         :volumeMounts [{:name "data"   :mountPath "/data"}
+                        #_{:name "config" :mountPath "/data/config.yaml" :subPath "config.yaml"}
+                        #_{:name "config" :mountPath "/data/registration.yaml" :subPath "registration.yaml"}]}]
+
+       :volumes
+       [{:name "data" :persistentVolumeClaim {:claimName "discord-bridge-data"}}
+        {:name "config" :configMap {:name "discord-bridge-config"}}]}}}}
+
+   :k8s:service-opts
+   {:spec {:selector {:app 'app-name}
+           :ports [{:name 'app-name :port 'port :targetPort 'port}]}}})

src/main/k8s/services/matrix/mmr/database/service.cljs

@@ -0,0 +1,37 @@
+(ns k8s.services.matrix.mmr.database.service)
+
+(def config
+  {:stack [:vault:prepare :k8s:pvc :k8s:deployment :k8s:service]
+   :app-namespace "matrix"
+   :app-name      "mmr-db"
+   
+   :k8s:pvc-opts
+   {:metadata {:name "mmr-pg-data"
+               :namespace "matrix"}
+    :spec {:storageClassName "hcloud-volumes"
+           :accessModes ["ReadWriteOnce"]
+           :resources {:requests {:storage "10Gi"}}}}
+
+   :k8s:deployment-opts
+   {:spec
+    {:template
+     {:metadata
+      {:annotations
+       {"backup.velero.io/backup-volumes" "db"}}
+      :spec
+      {:containers
+       [{:name 'app-name
+         :image "postgres:14-alpine"
+         :ports [{:containerPort 5432}]
+         :env [{:name "PGDATA" :value "/var/lib/postgresql/data/pgdata"}
+               {:name "POSTGRES_USER" :value 'username}
+               {:name "POSTGRES_PASSWORD" :value 'password}
+               {:name "POSTGRES_DB" :value 'db-name}]
+         :volumeMounts [{:name "db" :mountPath "/var/lib/postgresql/data"}]}]
+
+       :volumes
+       [{:name "db" :persistentVolumeClaim {:claimName "mmr-pg-data"}}]}}}}
+
+   :k8s:service-opts
+   {:spec {:selector {:app 'app-name}
+           :ports [{:name 'app-name :port 5432 :targetPort 5432}]}}})

src/main/k8s/services/matrix/mmr/service.cljs

@@ -2,20 +2,57 @@
 (ns k8s.services.matrix.mmr.service)
 
 (def config
-  {:stack [:vault-secrets :deployment :service :ingress]
+  {:stack [:vault:prepare [:k8s :config-map :deployment :service]]
    :image-port    80
    :app-namespace "matrix"
    :app-name      "matrix-media-repo"
-   :deployment-opts {:spec {:template {:spec {:containers [{:name 'app-name :image '(str repo "/" app-name ":v1.3.8")
-                                                            :volumeMounts [{:name "data" :mountPath "/data"}]}]
-                                              :initContainers [{:name "init-permissions"
-                                                                :image "busybox:latest"
-                                                                :command ["sh" "-c" "chown -R 1000:1000 /data"]
-                                                                :volumeMounts [{:name "data" :mountPath "/data"}]
-                                                                :securityContext {:runAsUser 0 :runAsGroup 0}}]
-                                              :volumes [{:name "data" :hostPath {:path "/opt/mmr/data" :type "DirectoryOrCreate"}}]}}}}})
+   :k8s:config-map-opts
+   {:metadata {:name "mmr-config"}
+    :data {"media-repo.yaml"
+           '(stringify
+             {:repo {:port port
+                     :bindAddress "0.0.0.0"
+                     :logLevel "debug"}
+              :database {:postgres db-login-url}
+              :homeservers [{:name homeserver
+                             :csApi (str "https://" homeserver)}]
+              :accessTokens {:appservices [{:id "discord"
+                                            :asToken discord-app-service-token
+                                            :senderUserId discord-send-user-id
+                                            :userNamespaces [{:regex user-namespace-regex}]}]}
+              :admins [admin]
+              :datastores [{:type "s3"
+                            :id s3-id
+                            :forKinds ["all"]
+                            :opts {:tempPath "/tmp/media-repo"
+                                   :endpoint s3-endpoint
+                                   :accessKeyId s3-access-key
+                                   :accessSecret s3-secret-key
+                                   :ssl true
+                                   :bucketName s3-bucket-name
+                                   :region s3-region}}]
+              :rateLimit {:enabled false}})}}
+   :k8s:deployment-opts
+   {:spec
+    {:template
+     {:spec
+      {:containers
+       [{:name 'app-name
+         :image '(str repo "/" app-name ":v1.3.8") 
+         :command ["/usr/local/bin/media_repo"] 
+         :args ["-config" "/data/media-repo.yaml"]
+   
+         :volumeMounts [{:name "config-vol"
+                         :mountPath "/data/media-repo.yaml"
+                         :subPath "media-repo.yaml"}
+   
+                        {:name "temp-vol"
+                         :mountPath "/tmp/media-repo"}]}]
+   
+       :volumes
+       [{:name "config-vol" :configMap {:name "mmr-config"}}
+        {:name "temp-vol" :emptyDir {}}]}}}}
+   })
 
 
-
-;;
-;;      - ./personal/matrix/mmr:/data
+   

src/main/k8s/services/matrix/turn/service.cljs

@@ -1,14 +1,67 @@
 (ns k8s.services.matrix.turn.service)
 
-;;     - ./personal/matrix/coturn.conf:/etc/coturn/turnserver.conf
+(defn generate-all-ports [start-relay end-relay]
+  (concat
+   [{:name "signaling-udp" :port 3478 :targetPort 3478 :nodePort 30478 :containerPort 3478 :protocol "UDP"}
+    {:name "signaling-tcp" :port 3478 :targetPort 3478 :nodePort 30478 :containerPort 3478 :protocol "TCP"}
+
+    {:name "tls-udp" :port 5349 :targetPort 5349 :nodePort 30549 :containerPort 5349 :protocol "UDP"}
+    {:name "tls-tcp" :port 5349 :targetPort 5349 :nodePort 30549 :containerPort 5349 :protocol "TCP"}]
+
+   (map (fn [p]
+          {:name (str "relay-" p)
+           :port p
+           :targetPort p
+           :nodePort p
+           :containerPort p
+           :protocol "UDP"})
+        (range start-relay (inc end-relay)))))
+
+(def all-ports (generate-all-ports 32000 32050))
 
 (def config
-  {:stack [:vault-secrets :docker-image :deployment :service :ingress]
-   :image-port     80
+  {:stack [:vault:prepare [:k8s :config-map :deployment :service]]
+   :image-port    nil
    :app-namespace "matrix"
    :app-name      "coturn"
-   :image-opts {:build {:args {:FOUNDRY_USERNAME 'FOUNDRY_USERNAME
-                               :FOUNDRY_PASSWORD 'FOUNDRY_PASSWORD}}
-                :imageName '(str repo "/" app-name ":latest")}
-   :deployment-opts {:spec {:template {:spec {:imagePullSecrets [{:name "harbor-creds-secrets"}]
-                                              :containers [{:name 'app-name :image '(str repo "/" app-name ":latest")}]}}}}})
+
+   :k8s:config-map-opts
+   {:metadata {:name "coturn-config"}
+    :data {"turnserver.conf"
+           '(str
+             "listening-port=3478\n"
+             "tls-listening-port=5349\n"
+             "min-port=32000\n"
+             "max-port=32050\n"
+
+             (str "external-ip=" public-ip "\n")
+
+             (str "realm" homeserver "\n")
+             (str "server-name=" host "\n")
+             "log-file=stdout\n"
+
+             "use-auth-secret\n"
+             (str "static-auth-secret=" secret-auth "\n")
+             "fingerprint\n"
+             "lt-cred-mech\n")}}
+
+   :k8s:deployment-opts
+   {:spec
+    {:template
+     {:spec
+      {:volumes [{:name "config" :configMap {:name "coturn-config"}}]
+       :containers [{:name 'app-name
+                     :image "coturn/coturn:latest"
+
+                     :ports (map #(select-keys % [:name :containerPort :protocol])
+                                 all-ports)
+
+                     :volumeMounts [{:name "config"
+                                     :mountPath "/etc/coturn/turnserver.conf"
+                                     :subPath "turnserver.conf"}]}]}}}}
+
+   :k8s:service-opts
+   {:spec {:type "NodePort"
+           :selector {:app 'app-name}
+           :ports (map #(select-keys % [:name :port :targetPort :nodePort :protocol])
+                       all-ports)}}})

src/main/k8s/services/mesite/service.cljs

@@ -8,7 +8,7 @@
   {:stack [:vault:prepare
            :harbor:robot-account
            :docker:image
-           [:k8s :namespace :deployment :service :httproute]] 
+           [:k8s :deployment :service :httproute]] 
    :app-name "mesite"
    :app-namespace "generic"
    :docker:image-opts {:context {:location "https://codeberg.org/Gigia/mesite.git"}

src/main/k8s/services/nextcloud/database/service.cljs

@@ -0,0 +1,40 @@
+(ns k8s.services.nextcloud.database.service)
+
+(def config
+  {:stack [:vault:prepare :k8s:pvc :k8s:deployment :k8s:service]
+   :app-namespace "nextcloud"
+   :app-name      "nextcloud-db"
+
+   :k8s:pvc-opts
+   {:metadata {:name "nextcloud-mariadb-disk"
+               :namespace "nextcloud"}
+    :spec {:storageClassName "hcloud-volumes"
+           :accessModes ["ReadWriteOnce"]
+           :resources {:requests {:storage "10Gi"}}}}
+
+   :k8s:deployment-opts
+   {:spec
+    {:template
+     {:metadata
+      {:annotations {"backup.velero.io/backup-volumes" "db"}}
+
+      :spec
+      {:containers
+       [{:name 'app-name
+         :image "mariadb:10.6"
+
+         :ports [{:containerPort 3306}]
+
+         :env [{:name "MYSQL_ROOT_PASSWORD" :value 'mariadb-root-password}
+               {:name "MYSQL_DATABASE" :value "nextcloud"}
+               {:name "MYSQL_USER" :value 'username}
+               {:name "MYSQL_PASSWORD" :value 'mariadb-password}]
+
+         :volumeMounts [{:name "db" :mountPath "/var/lib/mysql"}]}]
+
+       :volumes
+       [{:name "db" :persistentVolumeClaim {:claimName "nextcloud-mariadb-disk"}}]}}}}
+
+   :k8s:service-opts
+   {:spec {:selector {:app 'app-name}
+           :ports [{:name 'app-name :port 3306 :targetPort 3306}]}}})

src/main/k8s/services/nextcloud/service.cljs

@@ -1,17 +1,61 @@
 (ns k8s.services.nextcloud.service)
 
 (def config
-  {:stack [:vault-secrets :chart :ingress]
+  {:stack [:vault:prepare [:k8s :httproute :chart]]
    :app-namespace "nextcloud"
    :app-name      "nextcloud"
-   :image-port 8080
+   :image-port 80
    :vault-load-yaml true
-   :chart-opts {:fetchOpts {:repo "https://nextcloud.github.io/helm/"}
-                :values {:nextcloud {:host 'host
-                                     :trustedDomains ['host 'app-name]}}
-                :transformations (fn [args _opts]
-                                   (let [kind (get-in args [:resource :kind])]
-                                     (if (some #{kind} ["StatefulSet" "PersistentVolumeClaim" "Ingress"])
-                                       (update-in args [:resource :metadata :annotations]
-                                                  #(assoc (or % {}) "pulumi.com/skipAwait" "true"))
-                                       args)))}})
+   :k8s:chart-opts {:repositoryOpts {:repo "https://nextcloud.github.io/helm/"}
+                    :values
+                    {:podAnnotations {"backup.velero.io/backup-volumes" "data"}
+                     :trustedDomains ['host 'app-name]
+                     :nextcloud {:username 'username
+                                 :password 'password
+                                 :host 'host
+                                 :containerPort 80
+                                 :persistence {:enabled true
+                                               :storageClass "juicefs-sc"
+                                               :accessMode "ReadWriteMany"
+                                               :size "1Ti"}}
+                     :service {:port 80}
+                     :redis {:auth {:password 'redis-password}}
+                     :externalDatabase {:enabled true
+                                        :type "mysql"
+                                        :host "nextcloud-db.nextcloud.svc.cluster.local"
+                                        :database "nextcloud"
+                                        :user 'username
+                                        :password 'mariadb-password}
+                     :internalDatabase {:enabled false}
+                     :mariadb {:enabled false
+                               :auth {:username 'username
+                                      :password 'mariadb-password
+                                      :rootPassword 'mariadb-root-password}
+                               :architecture "standalone"
+                               :primary {:podAnnotations {"backup.velero.io/backup-volumes" "data"}
+                                         :persistence {:enabled true
+                                                       :storageClass "hcloud-volumes"
+                                                       :size "8Gi"}}
+                               ;; Obligatory what the fuck Broadcom, why are you like this. RIP Bitnami
+                               :volumePermissions {:enabled true
+                                                   :image {:registry "docker.io"
+                                                           :repository "bitnami/os-shell"
+                                                           :tag "latest"
+                                                           :pullPolicy "Always"}}}
+
+                     :transformations (fn [args _opts]
+                                        (let [kind (get-in args [:resource :kind])]
+                                          (if (some #{kind} ["StatefulSet" "PersistentVolumeClaim" "Ingress"])
+                                            (update-in args [:resource :metadata :annotations]
+                                                       #(assoc (or % {}) "pulumi.com/skipAwait" "true"))
+                                            args)))}}
+   :k8s:httproute-opts {:spec {::hostnames ['host]
+                               :rules [{:matches [{:path {:type "PathPrefix" :value "/"}}]
+                                        :filters [{:type "ResponseHeaderModifier"
+                                                   :responseHeaderModifier
+                                                   {:set [{:name "Content-Security-Policy"
+                                                           :value "frame-src 'self' https://cinny.hampter.quest https://productive.chickensalad.quest https://gitea.chickensalad.quest;"}]
+                                                    :remove ["X-Content-Security-Policy"]}}]
+
+                                        :backendRefs [{:name "nextcloud"
+                                                       :port 80}]}]}}})

src/main/k8s/services/prometheus/service.cljs

@@ -1,13 +1,11 @@
 (ns k8s.services.prometheus.service)
 
 (def config
-  {:stack [:vault-secrets :chart]
+  {:stack [:vault:prepare :k8s:chart]
    :app-namespace "prometheus"
    :app-name      "prometheus"
-   :image-port 8080
-   :vault-load-yaml true
-   :chart-opts {:chart "kube-prometheus-stack"
-                :fetchOpts {:repo "https://prometheus-community.github.io/helm-charts"}
+   :k8s:chart-opts {:chart "kube-prometheus-stack"
+                :repositoryOpts {:repo "https://prometheus-community.github.io/helm-charts"}
                 :namespace "monitoring"
                 :values {:grafana {:adminPassword 'password
                                    :ingress {:enabled true

src/main/k8s/services/renovate/service.cljs

@@ -8,7 +8,7 @@
    :image-port    8080
    :vault-load-yaml true
    :chart-opts
-   {:fetchOpts {:repo "https://docs.renovatebot.com/helm-charts"}
+   {:repositoryOpts {:repo "https://docs.renovatebot.com/helm-charts"}
     :values
     {:renovate
      {:config {:platform "github"

src/main/k8s/services/velero/service.cljs

@@ -0,0 +1,55 @@
+(ns k8s.services.velero.service)
+
+(def config
+  {:stack [:vault:prepare :k8s:secret :k8s:chart]
+   :app-namespace "velero"
+   :app-name      "velero"
+
+   :k8s:secret-opts
+   {:metadata {:name "velero-s3-creds"}
+    :stringData {"cloud" '(str "[default]\n"
+                               "aws_access_key_id = " s3-access-key "\n"
+                               "aws_secret_access_key = " s3-secret-key)}}
+
+   :k8s:chart-opts
+   {:repositoryOpts {:repo 'repo}
+    :chart "velero"
+    :version "5.1.0"
+    
+    :values
+    {:deployNodeAgent true
+
+     :configuration
+     {:backupStorageLocation
+      [{:name "default"
+        :provider "aws"
+        :bucket 's3-bucket-name
+        :config {:region 's3-region
+                 :s3ForcePathStyle true
+                 :s3Url 's3-url}}]
+
+      :volumeSnapshotLocation
+      [{:name "default"
+        :provider "aws"
+        :config {:region 's3-region}}]}
+
+
+     :credentials {:useSecret true
+                   :existingSecret "velero-s3-creds"}
+
+     :initContainers
+     [{:name "velero-plugin-for-aws"
+       :image "velero/velero-plugin-for-aws:v1.8.0"
+       :volumeMounts [{:mountPath "/target" :name "plugins"}]}]
+
+     :defaultVolumesToFsBackup true
+
+     :nodeAgent {:resources {:requests {:cpu "50m" :memory "64Mi"}
+                             :limits   {:cpu "1000m" :memory "1Gi"}}}
+
+     :schedules
+     {:daily-backup
+      {:disabled false
+       :schedule "0 4 * * *"
+       :template {:ttl "720h"
+                  :includedNamespaces ["matrix" "generic" "home" "nextcloud"]}}}}}})

src/main/service_registries.cljs

@@ -1,76 +0,0 @@
-(ns service-registries
-  (:require
-   [infra.init :as init]
-   [infra.openbao :as openbao]
-   [k8s.add-ons.csi-driver.hetzner :as hetzner-csi]
-   [infra.dns :as dns]
-   [infra.buildkit :as buildkit]
-   [k8s.preparers.harbor :as harbor-prepare]
-   
-   [k8s.add-ons.ingress-controller.caddy :as caddy]
-   [k8s.add-ons.gateway.traefik :as traefik]
-   [k8s.add-ons.cert-manager :as cert-manager]
-   [k8s.add-ons.csi-driver.wasabi :as wasabi-csi]
-   [k8s.add-ons.image-registry.harbor :as harbor]
-   [k8s.add-ons.secret-replicator :as secret-replicator]
-   [k8s.add-ons.minio :as minio]
-   [k8s.add-ons.s3proxy :as s3proxy]
-   [k8s.add-ons.proxy :as proxy]
-   [k8s.services.nextcloud.service :as nextcloud-service]
-   [k8s.services.mesite.service :as mesite-service]
-   [k8s.services.gitea.service :as gitea-service]
-   [k8s.services.act-runner.service :as act-runner-service]
-   [k8s.services.foundryvtt.service :as foundryvtt-service]
-   [k8s.services.productive.service :as productive-service]))
-
-(defn general-provider-output-refs []
-  {:vault  {:stack :init
-            :outputs ["vaultAddress" "vaultToken"]}
-   :harbor {:stack :shared
-            :outputs ["username" "password" "url"]}
-   :k8s   {:stack :base
-           :outputs ["kubeconfig"]}})
-
-(defn create-resource-definition [resource-configs stack-references provider-external-inputs]
-  {:resource-configs resource-configs
-   :stack-references stack-references
-   :provider-external-inputs provider-external-inputs})
-
-(def base-resources-definition
-  (create-resource-definition
-   [init/config]
-   nil
-   nil))
-
-(def initialize-resources-definition
-  (create-resource-definition
-   [hetzner-csi/config openbao/config]
-   ["base"]
-   {:k8s   {:stack :base
-            :outputs ["kubeconfig"]}}
-   ))
-
-(def shared-resources-definition
-  (create-resource-definition
-   [dns/config wasabi-csi/config proxy/config secret-replicator/config 
-    traefik/config cert-manager/config
-    harbor/config
-    ]
-   ["base" "init"]
-   (general-provider-output-refs)))
-
-(def preparation-resources-definition
-  (create-resource-definition
-   [harbor-prepare/config]
-   ["base" "init" "shared"]
-   (general-provider-output-refs)))
-
-
-(def deployment-resources-definition
-  (create-resource-definition
-   [#_buildkit/config #_nextcloud-service/config #_foundryvtt-service/config mesite-service/config #_productive-service/config #_gitea-service/config #_act-runner-service/config]
-   ["base" "init" "shared"]
-   (general-provider-output-refs)))
-
-
-(def deployment-matrix-service-registry [])

src/main/stack_resource_definitions.cljs

@@ -0,0 +1,121 @@
+(ns stack-resource-definitions
+  (:require
+   [infra.init :as init]
+   [infra.openbao :as openbao]
+   [k8s.add-ons.csi-driver.hetzner :as hetzner-csi]
+   [infra.dns :as dns]
+   [k8s.preparers.harbor :as harbor-prepare]
+   [k8s.add-ons.gateway.traefik :as traefik]
+   [k8s.add-ons.cert-manager :as cert-manager]
+   [k8s.add-ons.crd.cert-manager :as cert-manager-crd]
+   [k8s.add-ons.crd.gateway-api :as gateway-api-crd]
+   [k8s.add-ons.crd.traefik :as traefik-crds]
+   [k8s.add-ons.csi-driver.juicefs :as juicefs-csi]
+   [k8s.add-ons.csi-driver.wasabi :as wasabi-csi]
+   [k8s.add-ons.csi-driver.extra.redis :as redis-juicefs]
+   [k8s.add-ons.image-registry.harbor :as harbor]
+   [k8s.add-ons.secret-replicator :as secret-replicator]
+   [k8s.add-ons.proxy :as proxy]
+   [k8s.services.nextcloud.service :as nextcloud-service]
+   [k8s.services.nextcloud.database.service :as nextcloud-db-service]
+   [k8s.services.mesite.service :as mesite-service]
+   [k8s.services.gitea.service :as gitea-service]
+   [k8s.services.act-runner.service :as act-runner-service]
+   [k8s.services.foundryvtt.service :as foundryvtt-service]
+   [k8s.services.foundryvtt.service-2 :as girls-foundry-service]
+   [k8s.services.productive.service :as productive-service]
+   [k8s.services.velero.service :as velero-service]
+   [k8s.services.matrix.cinny.service :as cinny-service]
+   [k8s.services.matrix.element-call.service :as element-call-service]
+   [k8s.services.matrix.element.service :as element-service]
+   [k8s.services.matrix.element-call.livekit-server.service :as livekit-server-service]
+   [k8s.services.matrix.element-call.livekit-jwt.service :as livekit-jwt-service]
+   [k8s.services.matrix.mmr.service :as mmr-service]
+   [k8s.services.matrix.mmr.database.service :as mmr-db-service]
+   [k8s.services.matrix.mautrix-discord.database.service :as mautrix-discord-db-service]
+   [k8s.services.matrix.mautrix-discord.service :as mautrix-discord-service]
+   [k8s.services.matrix.home-server.well-known.service :as matrix-well-known-service]
+   [k8s.services.matrix.home-server.service :as homeserver-service]))
+
+(defn general-provider-output-refs []
+  {:vault  {:stack :init
+            :outputs ["vaultAddress" "vaultToken"]}
+   :harbor {:stack :shared
+            :outputs ["username" "password" "url"]}
+   :k8s   {:stack :base
+           :outputs ["kubeconfig"]}})
+
+(defn create-resource-definition [resource-configs stack-references provider-external-inputs]
+  {:resource-configs resource-configs
+   :stack-references stack-references
+   :provider-external-inputs provider-external-inputs})
+
+(def base-resources-definition
+  (create-resource-definition
+   [init/config]
+   nil
+   nil))
+
+(def initialize-resources-definition
+  (create-resource-definition
+   [hetzner-csi/config openbao/config]
+   ["base"]
+   {:k8s   {:stack :base
+            :outputs ["kubeconfig"]}}
+   ))
+
+(def shared-resources-definition
+  (create-resource-definition
+   [cert-manager-crd/config
+    gateway-api-crd/config
+    traefik-crds/config
+    dns/config
+    wasabi-csi/config proxy/config secret-replicator/config
+    redis-juicefs/config
+    juicefs-csi/config
+    cert-manager/config
+    traefik/config
+    harbor/config
+    ]
+   ["base" "init"]
+   (general-provider-output-refs)))
+
+(def preparation-resources-definition
+  (create-resource-definition
+   [harbor-prepare/config]
+   ["base" "init" "shared"]
+   (general-provider-output-refs)))
+
+
+(def deployment-resources-definition
+  (create-resource-definition
+   [girls-foundry-service/config foundryvtt-service/config 
+    mesite-service/config productive-service/config
+    nextcloud-db-service/config
+    nextcloud-service/config
+    velero-service/config
+    gitea-service/config 
+    ;;act-runner-service/config 
+    ]
+   ["base" "init" "shared"]
+   (general-provider-output-refs)))
+
+
+(def matrix-resources-definition 
+  (create-resource-definition
+   [cinny-service/config
+    element-call-service/config
+    element-service/config
+    livekit-server-service/config
+    livekit-jwt-service/config
+    mmr-db-service/config
+    mmr-service/config
+    mautrix-discord-db-service/config
+    mautrix-discord-service/config
+    matrix-well-known-service/config
+    homeserver-service/config
+
+    ]
+   ["base" "init" "shared"]
+   (general-provider-output-refs)))
+ 

src/main/utils/execution/general.clj

@@ -1,82 +0,0 @@
-(ns utils.execution.general
-  (:require
-   [clojure.walk]))
-
-(defmacro build-registry
-  "Generates a flat resource registry map from a nested provider definition map."
-  [definitions]
-  (let [map-entries
-        (reduce-kv
-         (fn [entries provider-group-kw provider-group-def]
-           (let [{:keys [root-sym provider-key resources]} provider-group-def
-                 provider-ns (name provider-group-kw)]
-             (reduce-kv
-              (fn [entries resource-kw resource-def]
-                (let [{:keys [path defaults-fn defaults-name]} resource-def
-                      resource-name (name resource-kw)
-                      final-key (keyword provider-ns resource-name)
-                      constructor-form (list* '.. root-sym path)
-                      defaults-fn-form
-                      (cond
-                        defaults-fn defaults-fn
-                        defaults-name (let [defaults-fn-sym (symbol "default" (name defaults-name))]
-                                        `(fn [env] (~defaults-fn-sym (:options env))))
-                        :else (let [defaults-fn-sym (symbol "default" resource-name)]
-                                `(fn [env] (~defaults-fn-sym (:options env)))))
-                      value-map `{:constructor ~constructor-form
-                                  :provider-key ~provider-key
-                                  :defaults-fn ~defaults-fn-form}]
-                  (conj entries [final-key value-map])))
-              entries
-              resources)))
-         []
-         definitions)]
-    `~(into {} map-entries)))
-
-
-(defn- p->-replace-percent
-  "Walks 'form' and replaces all instances of the symbol '%'
-   with the value of 'x'."
-  [form x]
-  (clojure.walk/postwalk
-    (fn [sub-form]
-      (if (= sub-form '%) x sub-form))
-    form))
-
-(defmacro pulet
-  "Sequential binding for Pulumi Outputs.
-   Looks like let*, but each binding is chained."
-  [bindings & body]
-  (if (empty? bindings)
-    `(do ~@body)
-    (let [[sym val & rest] bindings]
-      `(p-chain ~val
-                (fn [~sym]
-                  (pulet [~@rest] ~@body))))))
-
-
-
-
-(defmacro p-> [x & forms]
-  (let [wrap (fn [acc form]
-               (cond
-                 (map? form)
-                 `(p-chain ~acc (fn [val#] ~form))
-
-                 (keyword? form)
-                 `(p-chain ~acc #(get % ~form))
-
-                 (string? form)
-                 `(p-chain ~acc #(aget % ~form))
-
-                 (symbol? form)
-                 `(p-chain ~acc #(~form %))
-
-                 (list? form)
-                 `(p-chain ~acc ~form)
-                 
-
-                 :else
-                 (throw (ex-info "Unsupported form in p->" {:form form}))))]
-    (reduce wrap x forms)))
-

src/main/utils/execution/general.cljs

@@ -1,232 +0,0 @@
-(ns utils.execution.general (:require [clojure.walk :as walk]))
-
-
-(defn new-resource [resource-type resource-name final-args provider dependencies]
-  (let [base-opts (if (or (some? provider) (seq dependencies))
-                    {:enableServerSideApply false}  {})
-        opts (cond-> base-opts
-               (some? provider)     (assoc :provider provider)
-               (seq dependencies)   (assoc :dependsOn dependencies))]
-    (if (seq opts)
-      (new resource-type resource-name (clj->js final-args) (clj->js opts))
-      (new resource-type resource-name (clj->js final-args)))))
-
-(defn assoc-ins [m path-vals]
-  (reduce (fn [acc [path val]] (assoc-in acc path val)) m path-vals))
-
-(declare deep-merge)
-
-(defn merge-by-name
-  "Merges two vectors of maps by :name key."
-  [a b]
-  (let [a-map (into {} (map #(vector (:name %) %) a))
-        b-map (into {} (map #(vector (:name %) %) b))
-        merged (merge-with deep-merge a-map b-map)]
-    (vec (vals merged))))
-
-(defn deep-merge
-  "Recursively merges maps and intelligently merges vectors of maps by :name."
-  [a b]
-  (cond
-    (nil? b) a
-    (and (map? a) (map? b))
-    (merge-with deep-merge a b)
-
-    (and (vector? a) (vector? b)
-         (every? map? a) (every? map? b)
-         (some #(contains? % :name) (concat a b)))
-    (merge-by-name a b)
-    :else b))
-
-(defn make-transformer
-  "Given f that takes {:app-name .. :secrets ..}, where :secrets is a plain map
-   (already unwrapped inside .apply), return a Helm transformer."
-  [f]
-  (fn [{:keys [base-values app-name secrets]}]
-    (.apply secrets
-            (fn [smap]
-              (let [m (js->clj smap :keywordize-keys true)
-                    updates (f {:app-name app-name
-                                :secrets  m})
-                    after (clj->js (assoc-ins base-values updates))]
-                after)))))
-
-(defn make-paths [& path-groups]
-  (mapcat (fn [{:keys [paths backend]}]
-            (mapv (fn [p]
-                    {:path p
-                     :pathType "Prefix"
-                     :backend {:service backend}})
-                  paths))
-          path-groups))
-
-(defn generic-make-transformer
-  "Returns a Pulumi-compatible transformer that unwraps Output values via .apply."
-  [f {:keys [secrets base-values]}]
-  (.apply secrets
-          (fn [smap]
-            (let [m (js->clj smap :keywordize-keys true)
-                  updates (f {:function-keys m})
-                  result (clj->js (deep-merge base-values updates))]
-              result))))
-
-(defn safe-parse-int [s]
-  (let [n (js/parseInt s 10)]
-    (if (js/isNaN n) nil n)))
-
-(defn string->int? [s]
-  (and (string? s)
-       (re-matches #"^-?\d+$" s)))
-
-(defn- coerce-value [v]
-  (if (string->int? v)
-    (safe-parse-int v)
-    v))
-
-;; Whitelist functions for resolving templates. Intended to be extended.
-(def ^:private safe-fns
-  {'str str
-   'make-paths make-paths})
-
-(defn resolve-template [template values secondary-values]
-  (walk/postwalk
-   (fn [x]
-     (cond
-       (and (list? x) (contains? safe-fns (first x)))
-       (apply (get safe-fns (first x)) (rest x))
-       (symbol? x)
-       (if (contains? safe-fns x)
-         x
-         (let [kw (keyword x)]
-           (cond
-             (contains? values x) (coerce-value (get values x))
-             (contains? values kw) (coerce-value (get values kw))
-             (contains? secondary-values x) (coerce-value (get secondary-values x))
-             (contains? secondary-values kw) (coerce-value (get secondary-values kw))
-             :else x)))
-       :else x))
-   template))
-
-(defn generic-transform
-  "Takes a creator function and executes it with resolved arguments,
-   handling asynchronicity when secrets are present."
-  [creator-fn opts base-values secrets options]
-  (if (nil? secrets)
-    (let [final-args (clj->js (deep-merge base-values (resolve-template opts {} options)))]
-      (creator-fn final-args))
-    (.apply secrets
-            (fn [smap]
-              (let [m (js->clj smap :keywordize-keys true)
-                    final-args (clj->js (deep-merge base-values (resolve-template opts m options)))]
-                ;;(js/console.log final-args)
-                (creator-fn final-args))))))
-
-
-(defn resource-factory
-  [component-specs]
-  (fn [resource-type provider app-name dependencies opts]
-    (let [spec (get component-specs resource-type)
-          resource-class (:constructor spec)]
-      (if resource-class
-        (new-resource resource-class app-name opts provider dependencies)
-        (throw (js/Error. (str "Unknown resource type: " resource-type)))))))
-
-
-(defn component-factory [create-resource]
-  (fn [requested-components
-       resource-type
-       provider
-       app-name
-       dependencies
-       component-opts
-       defaults
-       secrets
-       options]
-
-    (when (requested-components resource-type)
-      (generic-transform
-       (fn [final-args]
-         (create-resource resource-type provider app-name dependencies final-args))
-       component-opts
-       defaults
-       secrets
-       options))))
-
-(defn deploy-stack-factory [func]
-  (fn [& args]
-    (let [[component-kws [options]] (split-with keyword? args)
-          requested-components (set component-kws)]
-      (func requested-components options))))
-
-(defn iterate-stack
-  [provider vault-data options secrets requested-components create-component-fn component-specs lifecycle-hooks]
-  (let [base-components
-        (reduce
-         (fn [acc [k {:keys [deps-fn opts-key defaults-fn]}]]
-           (let [env {:provider provider
-                      :options options
-                      :secrets secrets
-                      :components acc}
-                 app-name (get options :resource-name)
-                 deps     (deps-fn env)
-                 opts     (get options opts-key)
-                 defaults (defaults-fn env)
-                 component (create-component-fn requested-components k provider app-name deps opts defaults secrets options)]
-             (assoc acc k component)))
-         {:vault-secrets vault-data}
-         (select-keys component-specs requested-components))
-
-        final-components
-        (if lifecycle-hooks
-          (reduce
-           (fn [acc k]
-             (if-let [hook (get lifecycle-hooks k)]
-               (assoc acc k (hook {:options options
-                                   :components acc
-                                   :secrets secrets}))
-               acc))
-           base-components
-           requested-components)
-          base-components)]
-    final-components))
-
-(defn flatten-resource-groups
-  "Transforms a nested resource map into a flat, qualified-keyword map.
-   Example:
-   (flatten-resource-groups {:k8s {:chart {} :ingress {}}})
-   => {:k8s:chart {} :k8s:ingress {}}"
-  [config]
-  (into {}
-        (mapcat
-         (fn [[k v]]
-           (if (and (keyword? k) (map? v))
-             (map (fn [[inner-k inner-v]]
-                    [(keyword (name k) (name inner-k)) inner-v])
-                  v)
-             [[k v]]))
-         config)))
-
-
-(defn- is-output? [x] (some? (and x (.-__pulumiOutput x))))
-
-(defn p-apply-or-resolve
-  "Runtime helper. If 'v' is an Output, applies 'f' to it.
-   If 'v' is a plain value, calls 'f' with it."
-  [v f]
-  (if (is-output? v)
-    (.apply v f)
-    (f v)))
-
-(defn is-output? [x]
-  (some? (and x (.-__pulumiOutput x))))
-
-(defn p-chain [v f]
-  (if (is-output? v)
-    (.apply v f)
-    (f v)))
-
-(defn p-map [v f]
-  (p-chain v #(f %)))
-
-(defn p-lift [v]
- (if (is-output? v) v (js/Promise.resolve v)))

src/main/utils/execution/providers.cljs

@@ -1,130 +0,0 @@
-(ns utils.execution.providers
-  (:require
-   ["@pulumi/pulumi" :as pulumi] ["@pulumi/vault" :as vault] ["@pulumiverse/harbor" :as harbor] ["@pulumi/kubernetes" :as k8s]
-   [clojure.string :as str] [clojure.walk :as walk]
-   [utils.execution.general :refer [resolve-template]]
-   [utils.providers.k8s :as k8s-utils]
-   [utils.providers.harbor :as harbor-utils]
-   [utils.providers.docker :as docker-utils] [utils.providers.vault :as vault-utils]
-   [utils.execution.stack-processor :refer [deploy! component-specs]]))
-
-(defn resolve-provider-template [constructor name config]
-  {:constructor constructor
-   :name name
-   :config config})
-
-(def provider-templates
-  (into {} (map (fn [[k v]] [k (apply resolve-provider-template (vals v))])
-                {:vault vault-utils/provider-template
-                 :harbor harbor-utils/provider-template
-                 :k8s k8s-utils/provider-template})))
-
-(defn get-stack-refs [stack-ref-array]
-  (into {}
-        (map (fn [stack-name]
-               [(keyword stack-name)
-                (new pulumi/StackReference stack-name)])
-             stack-ref-array)))
-
-(defn extract-expanded-keywords [stack]
-  (let [expand-chain
-        (fn [chain]
-          (when (and (sequential? chain) (keyword? (first chain)))
-            (let [ns (or (namespace (first chain)) (name (first chain)))]
-              (map #(keyword ns (name %)) (rest chain)))))]
-
-    (mapcat (fn [item]
-              (cond
-                (and (sequential? item) (keyword? (first item)))
-                (expand-chain item)
-                (keyword? item)
-                [item]
-                :else
-                nil))
-            stack)))
-
-
-
-(defn get-all-providers [resource-configs]
-  (->> resource-configs
-       (mapcat (comp extract-expanded-keywords :stack))
-
-       (map (fn [component-key]
-              (if-let [ns (namespace component-key)]
-                (keyword ns)
-                (let [k-name (name component-key)
-                      parts (str/split k-name #":")]
-                  (when (> (count parts) 1)
-                    (keyword (first parts)))))))
-       (remove nil?)
-       (into #{})
-       vec))
-
-(def provider-rules
-  {:k8s k8s-utils/pre-deploy-rule})
-
-
-(defn provider-apply [stack-resources-definition pulumi-cfg]
-  (let [providers-needed (get-all-providers (:resource-configs stack-resources-definition))
-        provider-outputs-config (:provider-external-inputs stack-resources-definition)
-        stack-refs (get-stack-refs (:stack-references stack-resources-definition))
-        needed-output-configs (select-keys provider-outputs-config providers-needed)
-        ;; At some point we should add the ability for Providers to be passed Pulumi configs or our config map?
-        ;; Cloudflare and others may require or request a token.
-        outputs-to-fetch (reduce-kv
-                          (fn [acc _provider-key data]
-                            (let [stack-key (:stack data)
-                                  stack-ref (get stack-refs stack-key)
-                                  outputs (:outputs data)]
-
-                              (reduce
-                               (fn [m output-name]
-                                 (assoc m (keyword output-name) (.getOutput stack-ref output-name)))
-                               acc
-                               outputs)))
-                          {}
-                          needed-output-configs)
-
-        all-provider-inputs (pulumi/all (clj->js outputs-to-fetch))]
-
-    (.apply all-provider-inputs
-            (fn [values]
-              (js/Promise.
-               (fn [resolve _reject]
-                 (let [resolved-outputs (js->clj values :keywordize-keys true)
-                       instantiated-providers
-                       (reduce
-                        (fn [acc provider-key]
-                          (if-let [template (get provider-templates provider-key)]
-                            (let [constructor (:constructor template)
-                                  provider-name (:name template)
-                                  resolved-config (resolve-template (:config template) {} resolved-outputs)]
-
-                              (assoc acc provider-key (new constructor provider-name (clj->js resolved-config))))
-                            acc))
-                        {}
-                        providers-needed)
-                       pre-deploy-results
-                       (reduce-kv
-                        (fn [acc provider-key provider-instance]
-                          (if-let [rule-fn (get provider-rules provider-key)]
-                            (let [rule-results (rule-fn {:resource-configs (:resource-configs stack-resources-definition)
-                                                         :provider provider-instance})]
-                              (assoc acc provider-key rule-results))
-                            acc))
-                        {}
-                        instantiated-providers)]
-                   (resolve
-                    (deploy!
-                     {:pulumi-cfg pulumi-cfg
-                      :resource-configs (:resource-configs stack-resources-definition)
-                      :all-providers instantiated-providers
-                      :pre-deploy-deps pre-deploy-results})))))))))
-
-
-(defn execute [configs exports]
-  (->
-   (let [pulumi-cfg (pulumi/Config.)]
-     (provider-apply pulumi-cfg configs))
-   (exports)
-   (clj->js)))

src/main/utils/execution/safe_fns.cljs

@@ -1,48 +0,0 @@
-(ns utils.execution.safe-fns)
-
-(defn make-paths [& path-groups]
-  (mapcat (fn [{:keys [paths backend]}]
-            (mapv (fn [p]
-                    {:path p
-                     :pathType "Prefix"
-                     :backend {:service backend}})
-                  paths))
-          path-groups))
-
-
-(defn make-listeners [domains-or-json]
-  (let [domains (if (string? domains-or-json)
-                  (js->clj (js/JSON.parse domains-or-json))
-                  domains-or-json)]
-    (vec
-     (mapcat
-      (fn [domain]
-        (let [clean-name (clojure.string/replace domain #"\." "-")
-              secret-name (str clean-name "-tls")]
-
-          [{:name (str "https-root-" clean-name)
-            :port 8443
-            :protocol "HTTPS"
-            :hostname domain
-            :tls {:mode "Terminate"
-                  :certificateRefs [{:name secret-name}]}
-            :allowedRoutes {:namespaces {:from "All"}}
-            }
-
-           {:name (str "https-wild-" clean-name)
-            :port 8443
-            :protocol "HTTPS"
-            :hostname (str "*." domain)
-            :tls {:mode "Terminate"
-                  :certificateRefs [{:name secret-name}]}
-            :allowedRoutes {:namespaces {:from "All"}}
-            }]))
-      domains))))
-
-(def ^:public safe-fns
-  {'str str
-   'b64e (fn [s] (-> (.from js/Buffer s) (.toString "base64")))
-   'println #(js/console.log %)
-   'make-paths make-paths
-   'make-listeners make-listeners
-   'parse #(js->clj (js/JSON.parse %))})

src/main/utils/execution/stack_processor.cljs

@@ -1,396 +0,0 @@
-(ns utils.execution.stack-processor
-  (:require
-   ["@pulumi/kubernetes" :as k8s]
-   ["@local/crds/gateway" :as gateway-api]
-   ["@local/crds/cert_manager" :as cert-manager]
-   ["@pulumi/pulumi" :as pulumi]
-   ["@pulumi/vault" :as vault]
-   ["@pulumiverse/harbor" :as harbor]
-   [utils.providers.defaults :as default]
-   [utils.providers.vault :as vault-utils]
-   [utils.execution.general :refer [deep-merge new-resource resource-factory deploy-stack-factory iterate-stack]]
-   ["@pulumi/docker" :as docker]
-   ["@pulumi/docker-build" :as docker-build]
-   [clojure.walk :as walk]
-   [clojure.string :as str]
-   ["path" :as path]
-   [configs :refer [cfg]]
-   [utils.providers.k8s :as k8s-utils]
-   [utils.providers.harbor :as harbor-utils]
-   [utils.providers.docker :as docker-utils]
-   [utils.execution.safe-fns :refer [safe-fns]])
-  (:require-macros [utils.execution.general :refer [p-> build-registry]]))
-
-(defn safe-parse-int [s]
-  (let [n (js/parseInt s 10)]
-    (if (js/isNaN n) nil n)))
-
-(defn string->int? [s]
-  (and (string? s)
-       (re-matches #"^-?\d+$" s)))
-
-(defn- coerce-value [v]
-  (if (string->int? v)
-    (safe-parse-int v)
-    v))
-
-
-(defn- is-output? [x] (some? (.-__pulumiOutput x)))
-
-(defn resolve-template
-  [template secrets-map options-map]
-  (let [data (merge (js->clj options-map) (js->clj secrets-map))]
-    (walk/postwalk
-     (fn [x]
-       (cond
-         (and (list? x) (contains? safe-fns (first x)))
-         (let [f (get safe-fns (first x))
-               args (rest x)]
-           (if (some is-output? args)
-             (.apply (pulumi/all (clj->js args))
-                     (fn [resolved-args]
-                       (apply f (js->clj resolved-args))))
-             (apply f args)))
-         (and (list? x) (symbol? (first x)))
-         (cond
-           (= (first x) '->)
-           (let [[_ resource-key & steps] x
-                 resource (get data resource-key)]
-             (if-not resource x
-                     (reduce (fn [acc step]
-                               (cond
-                                 (and (symbol? step) (str/starts-with? (name step) ".-"))
-                                 (let [prop-name (subs (name step) 2)]
-                                   (aget acc prop-name))
-                                 (and (list? step) (= (first step) 'get))
-                                 (get acc (second step) (nth step 2 nil))
-                                 :else acc))
-                             resource
-                             steps)))
-
-           (= (first x) 'get) (get data (second x) (nth x 2 nil))
-           (= (first x) 'get-in) (get-in data (second x) (nth x 2 nil))
-           :else x)
-         (symbol? x)
-         (if (contains? safe-fns x)
-           x
-           (let [kw (keyword x)]
-             (cond
-               (contains? data x) (coerce-value (get data x))
-               (contains? data kw) (coerce-value (get data kw))
-               :else x)))
-         :else x))
-     template)))
-
-
-(defn generic-transform
-  "Takes a creator function and executes it with resolved arguments,
-   handling asynchronicity when secrets are present."
-  [creator-fn opts base-values secrets options]
-  (if (nil? secrets)
-    (let [final-args (clj->js (deep-merge base-values (resolve-template opts {} options)))]
-      (pulumi/output (creator-fn final-args)))
-    (.apply secrets
-            (fn [smap]
-              (let [m (js->clj smap :keywordize-keys true)
-                    final-args (clj->js (deep-merge base-values (resolve-template opts m options)))]
-                (creator-fn final-args))))))
-
-(def component-specs
-  {:vault {:provider-key :vault
-           :provider-deps [:k8s]}
-   ;; K8s Resources 
-   :k8s:namespace {:constructor (.. k8s -core -v1 -Namespace)
-                   :provider-key :k8s
-                   :defaults-fn (fn [env] ((get-in default/defaults [:k8s :namespace]) (:options env)))}
-
-   :k8s:secret {:constructor (.. k8s -core -v1 -Secret)
-                :provider-key :k8s
-                :defaults-fn (fn [env] ((get-in default/defaults [:k8s :secret]) (:options env)))}
-
-   :k8s:config-map {:constructor (.. k8s -core -v1 -ConfigMap)
-                    :provider-key :k8s
-                    :defaults-fn (fn [env] ((get-in default/defaults [:k8s :config-map]) (:options env)))}
-
-   :k8s:deployment {:constructor (.. k8s -apps -v1 -Deployment)
-                    :provider-key :k8s
-                    :defaults-fn (fn [env] ((get-in default/defaults [:k8s :deployment]) (:options env)))}
-
-   :k8s:service {:constructor (.. k8s -core -v1 -Service)
-                 :provider-key :k8s
-                 :defaults-fn (fn [env] ((get-in default/defaults [:k8s :service]) (:options env)))}
-
-   :k8s:ingress {:constructor (.. k8s -networking -v1 -Ingress)
-                 :provider-key :k8s
-                 :defaults-fn (fn [env] ((get-in default/defaults [:k8s :ingress]) (:options env)))}
-
-   :k8s:chart {:constructor (.. k8s -helm -v3 -Chart)
-               :provider-key :k8s
-               :defaults-fn (fn [env]
-                              (deep-merge ((get-in default/defaults [:k8s :chart]) (:options env))
-                                          (update-in (get-in (:options env) [:k8s:chart-opts]) [:values]
-                                                     #(deep-merge % (or (:yaml-values (:options env)) {})))))}
-   :k8s:storage-class {:constructor (.. k8s -storage -v1 -StorageClass)
-                       :provider-key :k8s
-                       :defaults-fn (fn [env] ((get-in default/defaults [:k8s :storage-class]) (:options env)))}
-
-   :k8s:pvc {:constructor (.. k8s -storage -v1 -PVC)
-             :provider-key :k8s
-             :defaults-fn (fn [env] ((get-in default/defaults [:k8s :pvc]) (:options env)))}
-
-   :k8s:gateway {:constructor (.. gateway-api -v1 -Gateway)
-                 :provider-key :k8s
-                 :defaults-fn (fn [env] ((get-in default/defaults [:k8s :gateway]) (:options env)))}
-
-   :k8s:httproute {:constructor (.. gateway-api -v1 -HTTPRoute)
-                   :provider-key :k8s
-                   :defaults-fn (fn [env] ((get-in default/defaults [:k8s :httproute]) (:options env)))}
-
-   :k8s:cluster-issuer {:constructor (.. cert-manager -v1 -ClusterIssuer)
-                        :provider-key :k8s
-                        :defaults-fn (fn [env] ((get-in default/defaults [:k8s :cluster-issuer]) (:options env)))} 
-
-   :k8s:certificates
-   {:constructor (.. cert-manager -v1 -Certificate)
-    :provider-key :k8s
-    :defaults-fn (fn [env]
-                   (let [{:keys [app-namespace is-prod?]} (:options env)]
-                    (p-> env :options :vault:prepare "stringData" .-domains
-                         #(vec
-                           (for [domain (js/JSON.parse %)] 
-                             (let [clean-name (clojure.string/replace domain #"\." "-")]
-                               {:_suffix clean-name
-                                :metadata {:namespace app-namespace}
-                                :spec {:dnsNames [domain (str "*." domain)]
-                                       :secretName (str clean-name "-tls")
-                                       :issuerRef {:name (if is-prod? "letsencrypt-prod" "letsencrypt-staging")
-                                                   :kind "ClusterIssuer"}}}))))))}
-
-   :k8s:certificate
-   {:constructor (.. cert-manager -v1 -Certificate)
-    :provider-key :k8s
-    :defaults-fn (fn [env] ((get-in default/defaults [:k8s :certificate]) (:options env)))}
-
-   ;; Docker Resources
-   :docker:image {:constructor (.. docker-build -Image)
-                  :provider-key :docker
-                  :defaults-fn (fn [env] ((get-in default/defaults [:docker :image]) (:options env)))}
-
-   ;; Harbor Resources
-   :harbor:project {:constructor (.. harbor -Project)
-                    :provider-key :harbor
-                    :defaults-fn (fn [env] ((get-in default/defaults [:harbor :project]) (:options env)))}
-
-   :harbor:robot-account {:constructor (.. harbor -RobotAccount)
-                          :provider-key :harbor
-                          :defaults-fn (fn [env] ((get-in default/defaults [:harbor :robot-account]) (:options env)))}})
-
-(defmulti deploy-resource
-  "Generic resource deployment multimethod.
-  Dispatches on the fully-qualified resource keyword.
-  Returns a map of {:resource (the-pulumi-resource) :common-opts-update {map-of-new-state}}."
-  (fn [dispatch-key _config] dispatch-key))
-
-(defmethod deploy-resource :default
-  [dispatch-key full-config]
-
-  (if-let [spec (get component-specs dispatch-key)]
-    (let [app-name       (:app-name full-config)
-          dependsOn      (:dependsOn full-config)
-          provider-key   (:provider-key spec)
-          provider       (get full-config provider-key)
-          resource-class (:constructor spec)
-          opts-key       (keyword (str (name dispatch-key) "-opts"))
-          component-opts (get full-config opts-key)
-          env            {:options full-config :secrets (:secrets full-config) :component-opts component-opts}
-          raw-defaults   (when-let [df (:defaults-fn spec)] (df env))]
-
-      (if resource-class
-        (let [base-creator (fn [final-args suffix]
-                             (let [final-name (if suffix
-                                                (str app-name "-" suffix)
-                                                app-name)]
-                               (new-resource resource-class
-                                             final-name
-                                             final-args
-                                             provider
-                                             dependsOn)))]
-
-          {:resource
-           (p-> raw-defaults
-                #(let [defaults-list (if (vector? %)
-                                       %
-                                       [%])
-                       is-multi?     (vector? %) resources
-                       (doall
-                        (map-indexed
-                         (fn [idx item]
-                           (let [suffix (cond
-                                          (:_suffix item) (:_suffix item)
-                                          is-multi? (str idx)
-                                          :else nil)
-                                 clean-item (dissoc item :_suffix)
-                                 item-creator (fn [resolved-args]
-                                                (base-creator resolved-args suffix))]
-
-                             (generic-transform item-creator
-                                                component-opts
-                                                clean-item
-                                                (:secrets env)
-                                                full-config)))
-                         defaults-list))]
-                   (if is-multi? resources (first resources))))})
-
-        (throw (js/Error. (str "No :constructor found for spec: " dispatch-key)))))
-
-    (throw (js/Error. (str "Unknown resource: " dispatch-key)))))
-
-(defmethod deploy-resource :vault:prepare
-  [_ config]
-  (let [prepare-opts (get config :vault:prepare-opts {})
-        defaults {:provider       (:k8s config)
-                  :vault-provider (:vault config)
-                  :app-name       (:app-name config)
-                  :app-namespace  (:app-namespace config)
-                  :load-yaml      (get config :vault-load-yaml false)}
-        final-args (merge defaults prepare-opts)
-        
-        prepared-vault-data (try
-                              (vault-utils/prepare final-args)
-                              (catch js/Error e
-                                (js/console.error "!!! Error in :vault:prepare :" e)
-                                nil))]
-    {:common-opts-update prepared-vault-data
-     :resource (:bind-secrets prepared-vault-data)}))
-
-(defmethod deploy-resource :vault:retrieve
-  [_ config]
-  (let [retrieve-opts (get config :vault:retrieve-opts {})
-        defaults {:vault-provider (:vault config)
-                  :app-name       (:app-name config)
-                  :app-namespace  (:app-namespace config)}
-        final-args (merge defaults retrieve-opts)
-        retrieved-data (try
-                         (vault-utils/retrieve (:vault-provider final-args)
-                                               (:app-name final-args)
-                                               (:app-namespace final-args))
-                         (catch js/Error e
-                           (js/console.error " Error in :vault:retrieve :" e)
-                           nil))]
-    {:common-opts-update retrieved-data}))
-
-;; https://www.pulumi.com/docs/iac/concepts/resources/dynamic-providers/
-(defmethod deploy-resource :generic:execute
-  [_ full-config]
-  (let [app-name       (:app-name full-config)
-        dependsOn      (:dependsOn full-config)
-        component-opts (assoc (:execute-opts full-config)
-                              :pulumi-cfg (:pulumi-cfg full-config)
-                              :secrets (:secrets full-config)
-                              )
-        defaults      {}
-        exec-fn        (:exec-fn full-config)
-        resource-id    (str app-name "-exec")
-        provider #js {:create (fn [inputs-js]
-                                (js/Promise.
-                                 (fn [resolve _reject]
-                                   (resolve
-                                    #js {:id resource-id
-                                         :outs inputs-js}))))
-                      :delete (fn [id old-inputs-js]
-                                (js/Promise.resolve))
-                      :update (fn [id old-inputs-js new-inputs-js]
-                                (js/Promise.
-                                 (fn [resolve _reject]
-                                     (resolve #js {:outs new-inputs-js}))))}
-        gen (generic-transform #(clj->js (exec-fn (js->clj % :keywordize-keys true))) component-opts defaults (:secrets full-config) full-config)
-        creator-fn (fn [inputs]
-                     (pulumi/dynamic.Resource.
-                      provider
-                      resource-id
-                      inputs
-                      (clj->js {:dependsOn (vec dependsOn)})))
-        resource (.apply gen #(creator-fn %))]
-    {:resource resource}))
-
-
-(defn handle-keyword-item [last-resource item config common-opts]
-  (let [dispatch-key item
-        depends-on (when last-resource [last-resource])
-        final-config (merge config common-opts {:dependsOn depends-on}) 
-        result-map (deploy-resource dispatch-key final-config)
-        resource (:resource result-map)
-        opts-update (:common-opts-update result-map)
-        resource-update (when resource {dispatch-key resource})]
-
-    [resource
-     resource-update
-     (merge common-opts opts-update resource-update)]))
-
-(defn handle-list-item [last-resource item config common-opts]
-  (let [provider-key (first item)
-        resource-keys (rest item)
-
-        nested-result
-        (reduce
-         (fn [nested-acc resource-key]
-           (let [inner-last-resource (get nested-acc :last-resource)
-                 inner-resources-map (get nested-acc :resources)
-                 inner-common-opts   (get nested-acc :common-opts)
-                 dispatch-key (keyword (str (name provider-key) ":" (name resource-key)))
-                 [new-resource new-resource-map new-common-opts]
-                 (handle-keyword-item inner-last-resource dispatch-key config inner-common-opts)]
-             {:last-resource (or new-resource inner-last-resource)
-              :resources     (merge inner-resources-map new-resource-map)
-              :common-opts   new-common-opts}))
-         {:last-resource last-resource
-          :resources     {}
-          :common-opts   common-opts}
-
-         resource-keys)]
-
-    [(:last-resource nested-result)
-     (:resources nested-result)
-     (:common-opts nested-result)]))
-
-(defn process-stack
-  "Recursively processes a stack configuration, building a dependency chain.
-  Returns a map of all created resources keyed by their dispatch keyword."
-  [stack-items config initial-common-opts]
-  (let [result
-        (reduce
-         (fn [acc item]
-           (let [{:keys [last-resource common-opts]} acc
-                 [new-resource new-resources new-common-opts]
-                 (if (keyword? item)
-                   (handle-keyword-item last-resource item config common-opts)
-                   (handle-list-item last-resource item config common-opts))]
-             {:last-resource (or new-resource last-resource)
-              :resources-map (merge (:resources-map acc) new-resources)
-              :common-opts new-common-opts}))
-         {:last-resource nil
-          :resources-map {}
-          :common-opts initial-common-opts}
-         stack-items)]
-    (:resources-map result)))
-
-
-(defn deploy! [{:keys [pulumi-cfg resource-configs all-providers]}]
-  (let [deployment-results
-        (into
-         {}
-         (for [config resource-configs]
-           (let [{:keys [stack app-name]} config
-                 _ (when (nil? config)
-                     (throw (js/Error. "Resource configs contain a nil value!")))
-
-                 common-opts (merge
-                              all-providers
-                              (select-keys config [:app-name :app-namespace])
-                              {:pulumi-cfg pulumi-cfg})]
-
-             [app-name (process-stack stack config common-opts)])))]
-    (clj->js deployment-results)))
-
-
-

src/main/utils/providers/defaults.cljs

@@ -1,12 +0,0 @@
-(ns utils.providers.defaults
-  (:require ["path" :as path]
-            [configs :refer [cfg]]
-            [utils.providers.k8s :as k8s]
-            [utils.providers.harbor :as harbor]
-            [utils.providers.docker :as docker]))
-
-
-(def defaults
-  {:k8s k8s/defaults
-   :harbor harbor/defaults
-   :docker docker/defaults})

src/main/utils/providers/docker.cljs

@@ -1,25 +0,0 @@
-(ns utils.providers.docker
-  (:require
-   [utils.execution.general :refer [generic-transform deep-merge new-resource component-factory resource-factory deploy-stack-factory iterate-stack]]
-   ["@pulumi/docker-build" :as docker]
-   ["path" :as path]
-   [configs :refer [cfg]]))
-
-(defn image [env]
-  (let [{:keys [app-name docker:image-opts]} env
-        context-path (.. path (join "." (-> cfg :resource-path)))
-        dockerfile-path (.. path (join context-path (str app-name ".dockerfile")))
-        base-args (if (:is-local docker:image-opts)
-                    {:context {:location context-path}
-                     :dockerfile {:location dockerfile-path}
-                     :imageName (str (-> cfg :docker-repo) "/" app-name ":latest")}
-                    {})]
-    base-args))
-
-(def defaults
-  {:image image})
-
-(def component-specs-defs
-  {:root-sym 'docker
-   :provider-key :harbor
-   :resources {:image {:path ['-Image]}}})

src/main/utils/providers/harbor.cljs

@@ -1,36 +0,0 @@
-(ns utils.providers.harbor
-  (:require
-   ["@pulumiverse/harbor" :as harbor]))
-
-(defn project [{:keys [app-name]}]
-  {:name app-name
-   :public false})
-
-(defn robot-account [{:keys [app-name]}]
-  {:name (str app-name "-robot")
-   :level "project"
-   :permissions [{:kind "project"
-                  :namespace app-name
-                  :access [{:action "push" :resource "repository"}
-                           {:action "pull" :resource "repository"}
-                           {:action "list" :resource "repository"}]}]})
-
-(def defaults
-  {:project project
-   :robot-account robot-account})
-
-(def provider-template
-  {:constructor (.. harbor -Provider)
-   :name "harbor-provider"
-   :config {:url      'url
-            :username 'username
-            :password 'password}})
-
-
-(def component-specs-defs
-  {:root-sym 'harbor
-   :provider-key :harbor
-   :resources
-   {:project       {:path ['-Project]}
-    :robot-account {:path ['-RobotAccount]
-                    :defaults-name 'robot}}})

src/main/utils/providers/k8s.cljs

@@ -1,198 +0,0 @@
-(ns utils.providers.k8s (:require ["@pulumi/kubernetes" :as k8s]))
-
-
-
-(defn cluster-issuer [{:keys [host is-prod?]}]
-  {:metadata {:name (if is-prod? "letsencrypt-prod" "letsencrypt-staging")}
-   :spec {:acme {:email "admin@example.com"
-                 :server (if is-prod? "https://acme-v02.api.letsencrypt.org/directory" "https://acme-staging-v02.api.letsencrypt.org/directory")
-                 :privateKeySecretRef {:name (if is-prod? "account-key-prod" "account-key-staging")}
-                 :solvers [{:dns01 {:cloudflare {:apiTokenSecretRef
-                                                 {:name "api-token-secret"
-                                                  :key "apiToken"}}}
-                            :selector {:dnsZones [(or host "example.com")]}}]}}})
-
-(defn certificate
-  [{:keys [app-name app-namespace host is-prod?]}]
-  (let [secret-name (str app-name "-cert")
-        domain (or host (str app-name ".example.com"))
-        issuer-name (if is-prod?
-                      "letsencrypt-prod"
-                      "letsencrypt-staging")]
-
-    {:apiVersion "cert-manager.io/v1"
-     :kind "Certificate"
-     :metadata {:name (str app-name "-cert")
-                :namespace app-namespace}
-     :spec {:secretName secret-name
-            :issuerRef {:name issuer-name
-                        :kind "ClusterIssuer"}
-            :dnsNames [domain]}}))
-
-
-(defn gateway
-  [{:keys [app-name]}]
-  {:apiVersion "gateway.networking.k8s.io/v1"
-   :kind "Gateway"
-   :metadata {:name "main-gateway"
-              :namespace "traefik"}
-   :spec {:gatewayClassName "traefik"
-          :listeners
-          [{:name "http"
-            :protocol "HTTP"
-            :port 80}
-           {:name "https"
-            :protocol "HTTPS"
-            :port 443
-            :tls {:certificateRefs
-                  [{:name (str app-name "-cert")
-                    :kind "Secret"}]}}]}})
-
-
-(defn httproute [{:keys [app-name app-namespace host]}]
-  {:apiVersion "gateway.networking.k8s.io/v1"
-   :kind "HTTPRoute"
-   :metadata {:name (str app-name "-route")
-              :namespace app-namespace}
-   :spec {:parentRefs [{:name "main-gateway"
-                        :namespace "traefik"}]
-          :hostnames [host]
-          :rules [{:matches [{:path {:type "PathPrefix"
-                                     :value "/"}}]
-                   :backendRefs [{:name app-name
-                                  :port 80}]}]}})
-
-(defn ingress [{:keys [app-name app-namespace host]}]
-  {:metadata {:name app-name
-              :namespace app-namespace}
-   :spec {:ingressClassName "caddy"
-          :rules [{:host host
-                   :http {:paths [{:path "/"
-                                   :pathType "Prefix"
-                                   :backend {:service {:name app-name
-                                                       :port {:number 80}}}}]}}]}})
-
-(defn chart [{:keys [app-name app-namespace]}]
-  {:chart     app-name
-   :namespace app-namespace
-   :transformations []})
-
-(defn config-map [{:keys  [app-name app-namespace]}]
-  {:metadata {:namespace app-namespace
-              :name app-name}
-   :data {}})
-
-(defn service [{:keys  [app-name app-namespace image-port]}]
-  {:metadata {:namespace app-namespace
-              :name app-name}
-   :spec {:selector {:app app-name}
-          :ports [{:port 80 :targetPort image-port}]}})
-
-(defn deployment [{:keys [app-name app-namespace image image-port]}]
-  {:metadata {:namespace app-namespace
-              :name app-name}
-   :spec {:selector {:matchLabels {:app app-name}}
-          :replicas 1
-          :template {:metadata {:labels {:app app-name}}
-                     :spec {:containers
-                            [{:name app-name
-                              :image image
-                              :ports [{:containerPort image-port}]}]}}}})
-
-
-(defn nspace [{:keys [app-namespace]}]
-  {:metadata {:name app-namespace}})
-
-(defn secret [{:keys [app-name app-namespace]}]
-  {:metadata {:name (str app-name "-secrets")
-              :namespace app-namespace}})
-
-(defn storage-class [{:keys [app-name]}]
-  {:metadata {:name app-name}})
-
-(def defaults
-  {:ingress       ingress
-   :gateway      gateway
-   :httproute    httproute
-   :certificate certificate
-   :cluster-issuer cluster-issuer
-   :chart         chart
-   :config-map    config-map
-   :service       service
-   :deployment    deployment
-   :namespace     nspace
-   :secret        secret
-   :storage-class storage-class})
-
-
-(def component-specs-defs
-  {:root-sym 'k8s
-   :provider-key :k8s
-   :resources
-   {:config-map {:path ['-core '-v1 '-ConfigMap]}
-    :storage-class {:path ['-core '-v1 '-StorageClass]}
-    :namespace  {:path ['-core '-v1 '-Namespace]}
-    :secret     {:path ['-core '-v1 '-Secret]}
-    :deployment {:path ['-apps '-v1 '-Deployment]}
-    :service    {:path ['-core '-v1 '-Service]}
-    :ingress    {:path ['-networking '-v1 '-Ingress]}
-    :chart      {:path ['-helm '-v3 '-Chart]
-                 :defaults-fn
-                 '(fn [env]
-                    (deep-merge (default/chart (:options env))
-                                (update-in (get-in (:options env) [:k8s:chart-opts]) [:values]
-                                           #(deep-merge % (or (:yaml-values (:options env)) {})))))}}})
-
-#_(def component-specs
-    :k8s:namespace {:constructor (.. k8s -core -v1 -Namespace)
-                    :provider-key :k8s
-                    :defaults-fn (fn [env] (defaults/namespace (:options env)))}
-
-    :k8s:secret {:constructor (.. k8s -core -v1 -Secret)
-                 :provider-key :k8s
-                 :defaults-fn (fn [env] (default/secret (:options env)))}
-
-    :k8s:deployment {:constructor (.. k8s -apps -v1 -Deployment)
-                     :provider-key :k8s
-                     :defaults-fn (fn [env] (default/deployment (:options env)))}
-
-    :k8s:service {:constructor (.. k8s -core -v1 -Service)
-                  :provider-key :k8s
-                  :defaults-fn (fn [env] (default/service (:options env)))}
-
-    :k8s:ingress {:constructor (.. k8s -networking -v1 -Ingress)
-                  :provider-key :k8s
-                  :defaults-fn (fn [env] (default/ingress (:options env)))}
-
-    :k8s:chart {:constructor (.. k8s -helm -v3 -Chart)
-                :provider-key :k8s
-                :defaults-fn (fn [env]
-                               (deep-merge (default/chart (:options env))
-                                           (update-in (get-in (:options env) [:k8s:chart-opts]) [:values]
-                                                      #(deep-merge % (or (:yaml-values (:options env)) {})))))})
-
-(def provider-template
-  {:constructor (.. k8s -Provider)
-   :name "k8s-provider"
-   :config {:kubeconfig 'kubeconfig}})
-
-
-(defn pre-deploy-rule
-  "k8s pre-deploy rule: scans the service registry and creates
-   all unique namespaces. Returns a map of created namespaces
-   keyed by their name."
-  [{:keys [resource-configs provider]}]
-  (let [namespaces (->> resource-configs
-                        (remove #(contains? % :no-namespace))
-                        (map :app-namespace)
-                        (remove nil?)
-                        (set))]
-    (into {}
-          (for [ns-name namespaces]
-            (let [resource-name ns-name
-                  ns-config {:metadata {:name resource-name
-                                        :namespace ns-name}}
-                  ns-resource (new (.. k8s -core -v1 -Namespace) resource-name
-                                   (clj->js ns-config)
-                                   (clj->js {:provider provider}))]
-              [ns-name ns-resource])))))

src/main/utils/providers/vault.cljs

@@ -1,71 +0,0 @@
-(ns utils.providers.vault
-  (:require
-   ["@pulumi/kubernetes" :as k8s]
-   ["@pulumi/pulumi" :as pulumi]
-   ["@pulumi/vault" :as vault]
-   ["fs" :as fs]
-   ["js-yaml" :as yaml]
-   ["path" :as path]
-   [configs :refer [cfg]]))
-
-(defn get-secret-val
-  "Extract a specific key from a Vault secret Output/Promise."
-  [secret-promise key]
-  (.then secret-promise #(aget (.-data %) key)))
-
-(defn initialize-mount [vault-provider vault-path service-name]
-  (let [service-secrets (into {} (get (-> cfg :secrets-json) (keyword service-name)))]
-    (new (.. vault -generic -Secret)
-         (str service-name "-secret")
-         (clj->js {:path (str vault-path)
-                   :dataJson (js/JSON.stringify (clj->js service-secrets))})
-         (clj->js {:provider vault-provider}))))
-
-(defn prepare
-  "Prepares common resources and values for a deployment from a single config map."
-  [config]
-  (let [{:keys [provider vault-provider app-name app-namespace load-yaml]} config
-        values-path (.join path js/__dirname ".." (-> cfg :resource-path) (str app-name ".yml"))]
-
-    (let [yaml-values (when load-yaml
-                        (js->clj (-> values-path
-                                     (fs/readFileSync "utf8")
-                                     (yaml/load))
-                                 :keywordize-keys true))
-          {:keys [secrets-data bind-secrets]}
-          (when vault-provider
-            (let [vault-path (str "secret/" app-name)
-                  _ (initialize-mount vault-provider vault-path app-name)
-                  secrets (pulumi/output (.getSecret (.-generic vault)
-                                                     (clj->js {:path vault-path})
-                                                     (clj->js {:provider vault-provider})))
-                  secrets-data (.apply secrets #(.. % -data))
-                  bind-secrets (when (and provider app-namespace)
-                                 (new (.. k8s -core -v1  -Secret) (str app-name "-secrets")
-                                      (clj->js {:metadata {:name (str app-name "-secrets")
-                                                           :namespace app-namespace}
-                                                :stringData secrets-data})
-                                      (clj->js {:provider provider})))]
-              {:secrets-data secrets-data
-               :bind-secrets bind-secrets}))]
-
-    {:secrets secrets-data
-     :yaml-path values-path
-     :yaml-values yaml-values
-     :bind-secrets bind-secrets})))
-
-
-(defn retrieve [vault-provider app-name]
-  (let [vault-path (str "secret/" app-name)
-        secrets (pulumi/output (.getSecret (.-generic vault)
-                                           (clj->js {:path vault-path})
-                                           (clj->js {:provider vault-provider})))
-        secrets-data (.apply secrets #(.. % -data))]
-    {:secrets secrets-data}))
-
-
-(def provider-template
-  {:constructor (.. vault -Provider)
-   :name "vault-provider"
-   :config {:address 'vaultAddress
-            :token   'vaultToken}})