Add config-file and gateway-class

README.md

@@ -1,3 +1,4 @@
+# Currently very much in alpha
 
 Initially this was a part of my Pulumi configs, but as the scope and usability expanded it made more sense for it to be a library instead.
 It's fully functional for what it seeks to do, which is provide an idiomatic way of expressing infrastructure as code through a familiar LISP style.
@@ -8,3 +9,65 @@ The best example of usage are [my own infrastructure configs](https://github.com
 
 
 I'll detail more later (check my config README if you're desperate to get this alpha build running).
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Future additions needed:
+- Extra CRDs aren't installed by default and currently have no mechanism to install them
+- There *must* be a way to declare duplicate resources in the same config group. Otherwise we create massive bloat without purpose. Config file resources or certificates being individually declarable several times is NOT a bad thing. This would provide greater fluidity. We still need to refine the multi-resource deployment, but certificates is a good start with the direction that should take.
+- Should also revise default-fn to recursively call certificate and just allow the default-fn to unwind the values.
+- Component spec really needs to be moved out of stack_processor as it is just such a large block of data that so better belongs w/ the providers themselves.
+- It may be helpful to redesign the stack mechanism entirely so that resources and such are declared like:
+    ```
+    (def config
+        {:stack [
+            {:item-name 
+                {:options-in-here}} 
+            {:item-name-2 
+                {:options-in-here}}
+        ]})
+    ```
+    Where this provides much clearer association and each resource has its options readily available. As such you could declare duplicate keys in the same config. It would make resource associations much more explicit and cleaner written.
+    It would require a decent amount of revision, so no rush on this.
+- Currently, certificates relies upon a prior step existing and that in itself is a bit of an anti-pattern... So in the future our options NEED some way of informing the resolver and deployer that it has custom execution.
+    ```
+    :k8s:certificates
+    {:constructor (.. cert-manager -v1 -Certificate)
+        :provider-key :k8s
+        :defaults-fn (fn [env]
+                    (p-> env :options :vault:prepare "stringData" .-domains
+                            #(vec
+                            (for [domain (js/JSON.parse %)] 
+                                (let [clean-name (clojure.string/replace domain #"\." "-")]
+                                {:_suffix clean-name
+                                :spec {:dnsNames [domain (str "*." domain)]
+                                        :secretName (str clean-name "-tls")}})))))}
+    ```
+    The above is unideal. I think the best path forward for that is an override?  Considering that some might not use Vault.
+    It might, instead, benefit from a high level user declaration of intent regarding the location of their secrets/settings. I mentioned above to have it resolve based on what providers utilized (within reason for support). That removes the inherent reliance, but it still does leave resolution in the default-fn in an unideal manner. It doesn't work to make top-level functions resolve on the outer layer as the Vault entry wouldn't exist yet. 
+    If we do the user intent, we can at least change it to be a standard such as 
+    ```
+    (p-> env :options :secrets .-domains #(function here))
+    ```
+    I should add that this function would be in a more *plugin* since it isn't inherently a built-in for K8s. Same for Gateway.
+    It wouldn't hurt to add some extension for developing these too. Increasing clarity on manner of declaration can not hurt.
+- Default values (like in K8) are opinionated. They do need to outline how to use a structure for example, but it should also be convenient to use any other resource like Nginx instead of Traefik or Azure instead of Cloudflare. A macro could be applied to them (preferably after their declaration, so their default state remains opinionated) to swap out which provider an individual chooses to use. It can be an added field in the core declarations for processing. Obvious goal for this is expansiveness. There should be clean, reusable defaults and everything should be easily modifiable and expandable.
+
+- Resource declarations might benefit from being *able* to splinter when needed. Currently they are VERY MUCH locked to a singleton pattern. While we can "loop" over stuff inside a declaration it still only ever makes *one* resource.
+- pulumi2crd should perhaps include some install instructions and some insight into usage
+Currently the script builds correctly, but since they are version dependant we might want to have some sort of version management for each of these components. That way they can be updated in a similar mechanism to a normal npm package. We can make CICD pipelines for them for this with some sort of cron scheduling. Emulating Renovate behavior (or we can see if Renovate can be useful here even). The script hosting repo only generates two CRDs, but it does outline the behavior well at least. Though locked into using an NPM package deployment isn't ideal. Those generated CRDs should not be baked into the provider utils but instead be treated as an expansion. This way it is neatly organized into official and extended functionality.
+- Local file/config loading or something should also be a provider, as obviously we would want to be able to pass through virtually anything to a service. That way they can be accessed later (this would replace the weird load-yaml that is a leftover from prior iterations)
+- Should add a Cloudflare provider at some point
+- I should also perhaps make loading in providers more automatic. Our defaulting is quite opinionated. It isn't inherently "bad" as it makes generation far simpler, but perhaps we have a base and a "read and expanded" variant available to the downstream.
+- Currently unable to effectively destructure secrets in the execute function in the current design. However, since we'd want to change to remove the anti-pattern mentioned above, we'd ideally actually just reference secrets through the vault resource output from the given resource config's stack execution.

deps.edn

@@ -1,2 +1,8 @@
-{:paths ["src/main"]
- :deps {funcool/promesa {:mvn/version "11.0.678"}}}
+{:paths ["src"]
+ :deps {funcool/promesa {:mvn/version "11.0.678"} 
+        thheller/shadow-cljs {:mvn/version "2.28.20"}
+        org.jboss.xnio/xnio-api {:mvn/version "3.8.8.Final"}
+        org.jboss.xnio/xnio-nio {:mvn/version "3.8.8.Final"}
+        org.jboss.logging/jboss-logging {:mvn/version "3.4.1.Final"}
+        org.wildfly.common/wildfly-common {:mvn/version "1.5.4.Final"}
+        org.jboss.threads/jboss-threads {:mvn/version "3.1.0.Final"}}}

src/main/providers/defaults.cljs

@@ -1,12 +0,0 @@
-(ns providers.defaults
-  (:require ["path" :as path]
-            [configs :refer [cfg]]
-            [providers.k8s :as k8s]
-            [providers.harbor :as harbor]
-            [providers.docker :as docker]))
-
-
-(def defaults
-  {:k8s k8s/defaults
-   :harbor harbor/defaults
-   :docker docker/defaults})

src/pulumicljs/execution/general.clj

@@ -1,4 +1,4 @@
-(ns execution.general
+(ns pulumicljs.execution.general
   (:require
    [clojure.walk]))
 

src/pulumicljs/execution/general.cljs

@@ -1,4 +1,4 @@
-(ns execution.general (:require [clojure.walk :as walk]))
+(ns pulumicljs.execution.general (:require [clojure.walk :as walk]))
 
 
 (defn new-resource [resource-type resource-name final-args provider dependencies]

src/pulumicljs/execution/providers.cljs

@@ -1,12 +1,12 @@
-(ns execution.providers
+(ns pulumicljs.execution.providers
   (:require
    ["@pulumi/pulumi" :as pulumi] ["@pulumi/vault" :as vault] ["@pulumiverse/harbor" :as harbor] ["@pulumi/kubernetes" :as k8s]
    [clojure.string :as str] [clojure.walk :as walk]
-   [execution.general :refer [resolve-template]]
-   [providers.k8s :as k8s-utils]
-   [providers.harbor :as harbor-utils]
-   [providers.docker :as docker-utils] [providers.vault :as vault-utils]
-   [execution.stack-processor :refer [deploy! component-specs]]))
+   [pulumicljs.execution.general :refer [resolve-template]]
+   [pulumicljs.providers.k8s :as k8s-utils]
+   [pulumicljs.providers.harbor :as harbor-utils]
+   [pulumicljs.providers.docker :as docker-utils] [pulumicljs.providers.vault :as vault-utils]
+   [pulumicljs.execution.stack-processor :refer [deploy! component-specs]]))
 
 (defn resolve-provider-template [constructor name config]
   {:constructor constructor
@@ -122,9 +122,9 @@
                       :pre-deploy-deps pre-deploy-results})))))))))
 
 
-(defn execute [configs exports]
+(defn execute [stack-resources-definition exports]
   (->
    (let [pulumi-cfg (pulumi/Config.)]
-     (provider-apply pulumi-cfg configs))
+     (provider-apply stack-resources-definition pulumi-cfg))
    (exports)
    (clj->js)))

src/pulumicljs/execution/safe_fns.cljs

@@ -1,4 +1,5 @@
-(ns execution.safe-fns)
+(ns pulumicljs.execution.safe-fns
+  (:require [clojure.string :as str]))
 
 (defn make-paths [& path-groups]
   (mapcat (fn [{:keys [paths backend]}]
@@ -17,7 +18,7 @@
     (vec
      (mapcat
       (fn [domain]
-        (let [clean-name (clojure.string/replace domain #"\." "-")
+        (let [clean-name (str/replace domain #"\." "-")
               secret-name (str clean-name "-tls")]
 
           [{:name (str "https-root-" clean-name)

src/pulumicljs/execution/stack_processor.cljs

@@ -1,25 +1,25 @@
-(ns execution.stack-processor
+(ns pulumicljs.execution.stack-processor
   (:require
    ["@pulumi/kubernetes" :as k8s]
-   ["@local/crds/gateway" :as gateway-api]
-   ["@local/crds/cert_manager" :as cert-manager]
+   ["pulumi-extra-crds/generated/crds/gateway" :as gateway-api]
+   ["pulumi-extra-crds/generated/crds/cert_manager" :as cert-manager]
    ["@pulumi/pulumi" :as pulumi]
    ["@pulumi/vault" :as vault]
    ["@pulumiverse/harbor" :as harbor]
-   [providers.defaults :as default]
-   [providers.vault :as vault-utils]
-   [execution.general :refer [deep-merge new-resource resource-factory deploy-stack-factory iterate-stack]]
+   [pulumicljs.providers.defaults :as default]
+   [pulumicljs.providers.vault :as vault-utils]
+   [pulumicljs.execution.general :refer [deep-merge new-resource resource-factory deploy-stack-factory iterate-stack]]
    ["@pulumi/docker" :as docker]
    ["@pulumi/docker-build" :as docker-build]
    [clojure.walk :as walk]
    [clojure.string :as str]
    ["path" :as path]
    [configs :refer [cfg]]
-   [providers.k8s :as k8s-utils]
-   [providers.harbor :as harbor-utils]
-   [providers.docker :as docker-utils]
-   [execution.safe-fns :refer [safe-fns]])
-  (:require-macros [execution.general :refer [p-> build-registry]]))
+   [pulumicljs.providers.k8s :as k8s-utils]
+   [pulumicljs.providers.harbor :as harbor-utils]
+   [pulumicljs.providers.docker :as docker-utils]
+   [pulumicljs.execution.safe-fns :refer [safe-fns]])
+  (:require-macros [pulumicljs.execution.general :refer [p-> build-registry]]))
 
 (defn safe-parse-int [s]
   (let [n (js/parseInt s 10)]
@@ -111,6 +111,10 @@
    :k8s:config-map {:constructor (.. k8s -core -v1 -ConfigMap)
                     :provider-key :k8s
                     :defaults-fn (fn [env] ((get-in default/defaults [:k8s :config-map]) (:options env)))}
+   
+   :k8s:config-file {:constructor (.. k8s -yaml -v2 -ConfigFile)
+                    :provider-key :k8s
+                    :defaults-fn (fn [env] ((get-in default/defaults [:k8s :config-file]) (:options env)))}
 
    :k8s:deployment {:constructor (.. k8s -apps -v1 -Deployment)
                     :provider-key :k8s
@@ -124,7 +128,7 @@
                  :provider-key :k8s
                  :defaults-fn (fn [env] ((get-in default/defaults [:k8s :ingress]) (:options env)))}
 
-   :k8s:chart {:constructor (.. k8s -helm -v3 -Chart)
+   :k8s:chart {:constructor (.. k8s -helm -v4 -Chart)
                :provider-key :k8s
                :defaults-fn (fn [env]
                               (deep-merge ((get-in default/defaults [:k8s :chart]) (:options env))
@@ -141,6 +145,10 @@
    :k8s:gateway {:constructor (.. gateway-api -v1 -Gateway)
                  :provider-key :k8s
                  :defaults-fn (fn [env] ((get-in default/defaults [:k8s :gateway]) (:options env)))}
+   
+   :k8s:gateway-class {:constructor (.. gateway-api -v1 -GatewayClass)
+                       :provider-key :k8s
+                       :defaults-fn (fn [env] ((get-in default/defaults [:k8s :gateway-class]) (:options env)))}
 
    :k8s:httproute {:constructor (.. gateway-api -v1 -HTTPRoute)
                    :provider-key :k8s

src/pulumicljs/providers/defaults.cljs

@@ -0,0 +1,12 @@
+(ns pulumicljs.providers.defaults
+  (:require ["path" :as path]
+            [configs :refer [cfg]]
+            [pulumicljs.providers.k8s :as k8s]
+            [pulumicljs.providers.harbor :as harbor]
+            [pulumicljs.providers.docker :as docker]))
+
+
+(def defaults
+  {:k8s k8s/defaults
+   :harbor harbor/defaults
+   :docker docker/defaults})

src/pulumicljs/providers/docker.cljs

@@ -1,6 +1,6 @@
-(ns providers.docker
+(ns pulumicljs.providers.docker
   (:require
-   [execution.general :refer [generic-transform deep-merge new-resource component-factory resource-factory deploy-stack-factory iterate-stack]]
+   [pulumicljs.execution.general :refer [generic-transform deep-merge new-resource component-factory resource-factory deploy-stack-factory iterate-stack]]
    ["@pulumi/docker-build" :as docker]
    ["path" :as path]
    [configs :refer [cfg]]))

src/pulumicljs/providers/harbor.cljs

@@ -1,4 +1,4 @@
-(ns providers.harbor
+(ns pulumicljs.providers.harbor
   (:require
    ["@pulumiverse/harbor" :as harbor]))
 

src/pulumicljs/providers/k8s.cljs

@@ -1,4 +1,4 @@
-(ns providers.k8s (:require ["@pulumi/kubernetes" :as k8s]))
+(ns pulumicljs.providers.k8s (:require ["@pulumi/kubernetes" :as k8s]))
 
 
 
@@ -35,19 +35,25 @@
   {:apiVersion "gateway.networking.k8s.io/v1"
    :kind "Gateway"
    :metadata {:name "main-gateway"
-              :namespace "traefik"}
-   :spec {:gatewayClassName "traefik"
+              :namespace app-name}
+   :spec {:gatewayClassName app-name
           :listeners
-          [{:name "http"
+          [#_{:name "http"
             :protocol "HTTP"
-            :port 80}
-           {:name "https"
+            :port 8000}
+           #_{:name "https"
             :protocol "HTTPS"
-            :port 443
+            :port 8443
             :tls {:certificateRefs
                   [{:name (str app-name "-cert")
                     :kind "Secret"}]}}]}})
 
+(defn gateway-class [{:keys [app-name]}]
+  {:apiVersion "gateway.networking.k8s.io/v1"
+   :kind "GatewayClass"
+   :metadata {:name app-name}
+   :spec {:controllerName "io.traefik/gateway"}})
+
 
 (defn httproute [{:keys [app-name app-namespace host]}]
   {:apiVersion "gateway.networking.k8s.io/v1"
@@ -110,14 +116,21 @@
 (defn storage-class [{:keys [app-name]}]
   {:metadata {:name app-name}})
 
+(defn config-file
+  [{:keys [app-name file]}]
+  {:name app-name
+   :properties {:file file}})
+
 (def defaults
   {:ingress       ingress
    :gateway      gateway
+   :gateway-class  gateway-class
    :httproute    httproute
    :certificate certificate
    :cluster-issuer cluster-issuer
    :chart         chart
    :config-map    config-map
+   :config-file config-file
    :service       service
    :deployment    deployment
    :namespace     nspace

src/pulumicljs/providers/vault.cljs

@@ -1,4 +1,4 @@
-(ns providers.vault
+(ns pulumicljs.providers.vault
   (:require
    ["@pulumi/kubernetes" :as k8s]
    ["@pulumi/pulumi" :as pulumi]