A Workaround for Syncing and Updating Multiple Repositories with Flux

Flux is the GitOps Kubernetes operator, which is most useful when used as a deployment tool at the end of a Continuous Delivery pipeline. Flux will make sure that your new container images and config changes are propagated to the cluster. However, at this moment, flux only works with a single git repository containing Kubernetes manifests. Let's say you have three applications from three different repositories. If you run ``fluxctl install`` for each application on different namespace, and list the controllers with the last namespace you created. ```bash fluxctl list-controllers --k8s-fwd-ns=app3 ``` ```bash WORKLOAD CONTAINER IMAGE RELEASE POLICY default:deployment/app1 app1 123456789123.dkr.ecr.ap-southeast-1.amazonaws.com/app1:f8ebcf87b02cd334b4228c1d22fe001dafff9ca6 ready default:deployment/app2 app2 123456789123.dkr.ecr.ap-southeast-1.amazonaws.com/app2:92218e4aeefa8f19f5e9a900bc7d07f38b8622c6 ready default:deployment/app3 app3 123456789123.dkr.ecr.ap-southeast-1.amazonaws.com/app3:a1a8231ff2ac89eb70fc353eeceb2470ee2d0ec3 ready automated ``` If you list the controllers with namespace ``app1`` ```bash fluxctl list-controllers --k8s-fwd-ns=app1 ``` There is no workload for it ```bash WORKLOAD CONTAINER IMAGE ``` Same as ``app1`` ```bash fluxctl list-controllers --k8s-fwd-ns=app2 ``` No workload is expected ```bash WORKLOAD CONTAINER IMAGE ``` Therefore, even you make a commit to repo ``app1`` or ``app2``, it never triggers the controller to sync and update the repo. Your deployment would remain unchanged. To fix it, run ```bash kubectl edit clusterrolebinding.rbac.authorization.k8s.io/flux ``` You should see ``` apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"rbac.authorization.k8s.io/v1beta1","kind":"ClusterRoleBinding","metadata":{"annotations":{},"labels":{"name":"flux"},"name":"flux"},"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"ClusterRole","name":"flux"},"subjects":[{"kind":"ServiceAccount","name":"flux","namespace":"app3"}]} creationTimestamp: "2020-03-13T16:31:43Z" labels: name: flux name: flux resourceVersion: "85027" selfLink: /apis/rbac.authorization.k8s.io/v1/clusterrolebindings/flux uid: 202463ba-6548-11ea-a8a2-025c790809a6 roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: flux subjects: - kind: ServiceAccount name: flux namespace: app3 ``` Since you create ``app3`` at the end, the cluster role binding config is modified when you run ``fluxctl install``. ``` clusterrolebinding.rbac.authorization.k8s.io/flux configured ``` If you check out flux RBAC template, you can see there is only one subject. ``` apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: name: {{ template "flux.clusterRoleName" . }} labels: app: {{ template "flux.name" . }} chart: {{ template "flux.chart" . }} release: {{ .Release.Name }} heritage: {{ .Release.Service }} roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: {{ template "flux.clusterRoleName" . }} subjects: - name: {{ template "flux.serviceAccountName" . }} namespace: {{ .Release.Namespace | quote }} kind: ServiceAccount {{- end -}} {{- end -}} ``` Therefore, to allow three applications at the same time, we need to add the missing two. ``` apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"rbac.authorization.k8s.io/v1beta1","kind":"ClusterRoleBinding","metadata":{"annotations":{},"labels":{"name":"flux"},"name":"flux"},"roleRef":{"apiGroup":"rbac.authorization.k8s.io","kind":"ClusterRole","name":"flux"},"subjects":[{"kind":"ServiceAccount","name":"flux","namespace":"app1"}]} creationTimestamp: "2020-03-13T16:31:43Z" labels: name: flux name: flux resourceVersion: "85027" selfLink: /apis/rbac.authorization.k8s.io/v1/clusterrolebindings/flux uid: 202463ba-6548-11ea-a8a2-025c790809a6 roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: flux subjects: - kind: ServiceAccount name: flux namespace: app1 - kind: ServiceAccount name: flux namespace: app2 - kind: ServiceAccount name: flux namespace: app3 ``` Once you save the file, it will update the config in the background. Now we can verify the result. ```bash fluxctl list-controllers --k8s-fwd-ns=app1 ``` ```bash WORKLOAD CONTAINER IMAGE RELEASE POLICY default:deployment/app1 app1 123456789123.dkr.ecr.ap-southeast-1.amazonaws.com/app1:f8ebcf87b02cd334b4228c1d22fe001dafff9ca6 ready automated default:deployment/app2 app2 123456789123.dkr.ecr.ap-southeast-1.amazonaws.com/app2:92218e4aeefa8f19f5e9a900bc7d07f38b8622c6 ready default:deployment/app3 app3 123456789123.dkr.ecr.ap-southeast-1.amazonaws.com/app3:a1a8231ff2ac89eb70fc353eeceb2470ee2d0ec3 ready ``` ```bash fluxctl list-controllers --k8s-fwd-ns=app2 ``` ```bash WORKLOAD CONTAINER IMAGE RELEASE POLICY default:deployment/app1 app1 123456789123.dkr.ecr.ap-southeast-1.amazonaws.com/app1:f8ebcf87b02cd334b4228c1d22fe001dafff9ca6 ready default:deployment/app2 app2 123456789123.dkr.ecr.ap-southeast-1.amazonaws.com/app2:92218e4aeefa8f19f5e9a900bc7d07f38b8622c6 ready automated default:deployment/app3 app3 123456789123.dkr.ecr.ap-southeast-1.amazonaws.com/app3:a1a8231ff2ac89eb70fc353eeceb2470ee2d0ec3 ready ``` ```bash fluxctl list-controllers --k8s-fwd-ns=app3 ``` ```bash WORKLOAD CONTAINER IMAGE RELEASE POLICY default:deployment/app1 app1 123456789123.dkr.ecr.ap-southeast-1.amazonaws.com/app1:f8ebcf87b02cd334b4228c1d22fe001dafff9ca6 ready default:deployment/app2 app2 123456789123.dkr.ecr.ap-southeast-1.amazonaws.com/app2:92218e4aeefa8f19f5e9a900bc7d07f38b8622c6 ready default:deployment/app3 app3 123456789123.dkr.ecr.ap-southeast-1.amazonaws.com/app3:a1a8231ff2ac89eb70fc353eeceb2470ee2d0ec3 ready automated ``` Then when you make a commit to your repo ``app1``, ``app2`` and ``app3``, it should auto release your application and your deployment.yml should be updated by flux with a latest docker image URI.