diff --git a/docs/developer-guide/README.md b/docs/developer-guide/README.md index 086d0f3..03efcc5 100644 --- a/docs/developer-guide/README.md +++ b/docs/developer-guide/README.md @@ -64,41 +64,34 @@ helm install clickhouse clickhouse --repo https://helm.altinity.com \
Object Storage -Add the minio operator +Create a secret for RustFS credentials ```sh -helm install minio-operator operator \ - --repo https://operator.min.io/ \ - --set operator.env\[0\].name=WATCHED_NAMESPACE \ - --set operator.env\[0\].value=currents \ - --set operator.replicaCount=1 +kubectl create secret generic currents-rustfs-user \ + --from-literal=RUSTFS_ACCESS_KEY=$(head -c 512 /dev/urandom | LC_ALL=C tr -cd 'a-zA-Z0-9' | head -c 32) \ + --from-literal=RUSTFS_SECRET_KEY=$(head -c 512 /dev/urandom | LC_ALL=C tr -cd 'a-zA-Z0-9' | head -c 32) ``` -Create the root user config (edit the username/password in samples/minio-config.env) +Install RustFS ```sh -kubectl create secret generic currents-minio-env-configuration --from-file=config.env=samples/minio-config.env +helm install rustfs rustfs --repo https://charts.rustfs.com -f samples/rustfs-helm-config.yaml ``` -Create the additional users for currents +Create the `currents` bucket ```sh -kubectl create secret generic currents-minio-user --from-literal=CONSOLE_ACCESS_KEY=$(head -c 512 /dev/urandom | LC_ALL=C tr -cd 'a-zA-Z0-9' | LC_ALL=C tr -dc 'a-zA-Z0-9' | head -c 32) --from-literal=CONSOLE_SECRET_KEY=$(head -c 512 /dev/urandom | LC_ALL=C tr -cd 'a-zA-Z0-9' | head -c 32) +kubectl apply -f samples/rustfs-create-bucket-job.yaml +kubectl wait --for=condition=complete job/rustfs-create-bucket --timeout=60s ``` -Create a minio tenant instance +Create an ingress for RustFS ```sh -helm install tenant tenant --repo https://operator.min.io/ -f samples/minio-tenant-helm-config.yaml +kubectl apply -f samples/local/rustfs-ingress.yaml ``` -Create an ingress for minio - -```sh -kubectl apply -f samples/local/minio-ingress.yaml -``` - -Note that you will need to add `mino.localhost` to your `/etc/hosts` file on the loopback +Note that you will need to add `rustfs.localhost` to your `/etc/hosts` file on the loopback
diff --git a/docs/eks/dependencies.md b/docs/eks/dependencies.md index d34e987..34da0a0 100644 --- a/docs/eks/dependencies.md +++ b/docs/eks/dependencies.md @@ -119,105 +119,93 @@ Follow this step if you plan to use provider (S3, Cloudflare) object storage (re ### Alternative Object Storage (in cluster) -Install Minio if you don't have access to an Object Storage provider (S3, Cloudflare). You will need an additional subdomain for Minio. +Install RustFS if you don't have access to an Object Storage provider (S3, Cloudflare). You will need an additional subdomain for RustFS. -Creates a single Pod instance of Minio with 10Gb of storage. +Creates a single Pod instance of RustFS with 10Gi of storage. -1. Add the minio operator +1. Create a secret for RustFS credentials ```sh - helm install minio-operator operator \ - --repo https://operator.min.io/ \ - --set operator.env\[0\].name=WATCHED_NAMESPACE \ - --set operator.env\[0\].value=currents \ - --set operator.replicaCount=1 + kubectl create secret generic currents-rustfs-user \ + --from-literal=RUSTFS_ACCESS_KEY=$(head -c 512 /dev/urandom | LC_ALL=C tr -cd 'a-zA-Z0-9' | head -c 32) \ + --from-literal=RUSTFS_SECRET_KEY=$(head -c 512 /dev/urandom | LC_ALL=C tr -cd 'a-zA-Z0-9' | head -c 32) ``` -2. Create the root user config environment - ```sh - printf 'export MINIO_ROOT_USER="%s"\nexport MINIO_ROOT_PASSWORD="%s"\n' $(head -c 512 /dev/urandom | LC_ALL=C tr -cd 'a-zA-Z0-9' | LC_ALL=C tr -dc 'a-zA-Z0-[B9' | head -c 32) $(head -c 512 /dev/urandom | LC_ALL=C tr -cd 'a-zA-Z0-9' | head -c 32) > minio-config.env - kubectl create secret generic currents-minio-env-configuration --from-file=config.env=minio-config.env - ``` - -3. Create the additional users for currents - ```sh - kubectl create secret generic currents-minio-user --from-literal=CONSOLE_ACCESS_KEY=$(head -c 512 /dev/urandom | LC_ALL=C tr -cd 'a-zA-Z0-9' | LC_ALL=C tr -dc 'a-zA-Z0-9' | head -c 32) --from-literal=CONSOLE_SECRET_KEY=$(head -c 512 /dev/urandom | LC_ALL=C tr -cd 'a-zA-Z0-9' | head -c 32) - ``` - -4. Create a Minio Tenant Values file +2. Create a RustFS values file - `minio-tenant-helm-config.yaml` + `rustfs-helm-config.yaml` ```yaml - tenant: - name: currents-minio - configSecret: - name: currents-minio-env-configuration - existingSecret: true - accessKey: null - secretKey: null - pools: - - servers: 1 - name: pool-0 - volumesPerServer: 1 - size: 10Gi - storageAnnotations: { } - storageLabels: { } - annotations: { } - labels: { } - tolerations: [ ] - nodeSelector: { } - affinity: { } - resources: { } - securityContext: - runAsUser: 1000 - runAsGroup: 1000 - fsGroup: 1000 - fsGroupChangePolicy: "OnRootMismatch" - runAsNonRoot: true - containerSecurityContext: - runAsUser: 1000 - runAsGroup: 1000 - runAsNonRoot: true - allowPrivilegeEscalation: false - capabilities: - drop: - - ALL - seccompProfile: - type: RuntimeDefault - topologySpreadConstraints: [ ] - buckets: - - name: currents - users: - - name: currents-minio-user + # Standalone mode for dev/test - single pod + mode: + standalone: + enabled: true + distributed: + enabled: false + + # Use the secret we created for credentials + secret: + existingSecret: "currents-rustfs-user" + + # Service configuration + service: + type: ClusterIP + endpoint: + port: 9000 + console: + port: 9001 + + # Disable gateway API / TraefikService CRD creation + gatewayApi: + gatewayClass: "" + + # Disable built-in ingress (we create our own for full control) + ingress: + enabled: false + + # Storage configuration + storageclass: + name: "" # Uses default storage class + dataStorageSize: "10Gi" + logStorageSize: "256Mi" + + # Resource limits + resources: + limits: + cpu: "500m" + memory: "512Mi" + requests: + cpu: "100m" + memory: "128Mi" ``` -5. Install the Minio Tenant Instance +3. Install RustFS ```sh - helm install minio-tenant tenant --repo https://operator.min.io/ -f minio-tenant-helm-config.yaml + helm install rustfs rustfs --repo https://charts.rustfs.com -f rustfs-helm-config.yaml ``` -6. Create an Ingress Resource to expose the Minio S3 api +4. Create an Ingress Resource to expose the RustFS S3 API Be sure to customize the following: - `alb.ingress.kubernetes.io/certificate-arn` - `spec.ingressClassName` - - `spec.rules.host` + - `spec.rules[0].host` - file: `minio-eks-ingress.yaml` + `rustfs-eks-ingress.yaml` ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: - name: ingress-minio + name: ingress-rustfs annotations: # Set to 'internet-facing' to expose to the public alb.ingress.kubernetes.io/scheme: internal alb.ingress.kubernetes.io/group.name: currents - # Set the ARN a resource managed by aws certificate manager, that matches the DNS host + # Set the ARN to a resource managed by AWS Certificate Manager alb.ingress.kubernetes.io/certificate-arn: "arn:aws:acm:" alb.ingress.kubernetes.io/target-type: ip - alb.ingress.kubernetes.io/backend-protocol: HTTPS - alb.ingress.kubernetes.io/healthcheck-protocol: HTTPS - alb.ingress.kubernetes.io/success-codes: '200,403' + alb.ingress.kubernetes.io/backend-protocol: HTTP + alb.ingress.kubernetes.io/healthcheck-protocol: HTTP + alb.ingress.kubernetes.io/healthcheck-path: /health + alb.ingress.kubernetes.io/success-codes: '200' spec: ingressClassName: alb-currents rules: @@ -229,14 +217,53 @@ Creates a single Pod instance of Minio with 10Gb of storage. pathType: Prefix backend: service: - name: minio + name: rustfs port: - number: 443 + number: 9000 + ``` + + ```sh + kubectl apply -f rustfs-eks-ingress.yaml + ``` + +5. Create the `currents` bucket by applying a Job that uses mc (MinIO client) + + `rustfs-create-bucket-job.yaml` + ```yaml + apiVersion: batch/v1 + kind: Job + metadata: + name: rustfs-create-bucket + spec: + ttlSecondsAfterFinished: 300 + template: + spec: + restartPolicy: Never + containers: + - name: mc + image: minio/mc:latest + env: + - name: RUSTFS_ACCESS_KEY + valueFrom: + secretKeyRef: + name: currents-rustfs-user + key: RUSTFS_ACCESS_KEY + - name: RUSTFS_SECRET_KEY + valueFrom: + secretKeyRef: + name: currents-rustfs-user + key: RUSTFS_SECRET_KEY + command: + - /bin/sh + - -c + - | + mc alias set rustfs http://rustfs:9000 $RUSTFS_ACCESS_KEY $RUSTFS_SECRET_KEY + mc mb --ignore-existing rustfs/currents ``` -7. Apply the Ingress file ```sh - kubectl apply -f minio-eks-ingress.yaml + kubectl apply -f rustfs-create-bucket-job.yaml + kubectl wait --for=condition=complete job/rustfs-create-bucket --timeout=60s ``` ### SMTP Email diff --git a/docs/eks/quickstart.md b/docs/eks/quickstart.md index 6f484fe..db81492 100644 --- a/docs/eks/quickstart.md +++ b/docs/eks/quickstart.md @@ -1,6 +1,6 @@ # Quickstart: Installing Currents on EKS -The QuickStart for installing the Currents Helm Chart on EKS runs through the commands to get the Currents App setup. This includes installing dependencies like MongoDB, Elasticsearch, and Minio into your Kubernetes namespace. +The QuickStart for installing the Currents Helm Chart on EKS runs through the commands to get the Currents App setup. This includes installing dependencies like MongoDB, ClickHouse, and RustFS into your Kubernetes namespace. ## Accessing Currents Docker Images @@ -104,14 +104,14 @@ Configure and install the Currents Helm Chart once all the services are ready. # NOTE: Choose either IAM (remove secretName) OR secret-based authentication. # Do NOT leave secretName in your configuration if you haven't created the secret. - # Option 3: For minio deployed in the same K8s namespace - # Use the following settings instead if you setup Minio - # secretName: currents-minio-user - # secretIdKey: CONSOLE_ACCESS_KEY - # secretAccessKey: CONSOLE_SECRET_KEY - # Set the endpoint to your Minio Route + # Option 3: For RustFS deployed in the same K8s namespace + # Use the following settings instead if you setup RustFS + # secretName: currents-rustfs-user + # secretIdKey: RUSTFS_ACCESS_KEY + # secretAccessKey: RUSTFS_SECRET_KEY + # Set the endpoint to your RustFS route # endpoint: https://storage.eks.example.com - # internalEndpoint: https://minio + # internalEndpoint: http://rustfs # bucket: currents # pathStyle: true diff --git a/samples/eks/eks-config.yaml b/samples/eks/eks-config.yaml index f02f178..818655a 100644 --- a/samples/eks/eks-config.yaml +++ b/samples/eks/eks-config.yaml @@ -34,10 +34,10 @@ currents: host: clickhouse-clickhouse objectStorage: endpoint: https://storage.eks.currents-sandbox.work - internalEndpoint: https://minio - secretName: currents-minio-user - secretIdKey: CONSOLE_ACCESS_KEY - secretAccessKey: CONSOLE_SECRET_KEY + internalEndpoint: http://rustfs + secretName: currents-rustfs-user + secretIdKey: RUSTFS_ACCESS_KEY + secretAccessKey: RUSTFS_SECRET_KEY bucket: currents pathStyle: true diff --git a/samples/eks/minio-eks-ingress.yaml b/samples/eks/minio-eks-ingress.yaml deleted file mode 100644 index 18af20b..0000000 --- a/samples/eks/minio-eks-ingress.yaml +++ /dev/null @@ -1,25 +0,0 @@ -apiVersion: networking.k8s.io/v1 -kind: Ingress -metadata: - name: ingress-minio - annotations: - alb.ingress.kubernetes.io/scheme: internet-facing - alb.ingress.kubernetes.io/group.name: currents - alb.ingress.kubernetes.io/certificate-arn: "arn:aws:acm:us-east-1:761136292957:certificate/0d3489f4-6b3e-4651-9538-4f6377fadeaa" - alb.ingress.kubernetes.io/target-type: ip - alb.ingress.kubernetes.io/backend-protocol: HTTPS - alb.ingress.kubernetes.io/healthcheck-protocol: HTTPS - alb.ingress.kubernetes.io/success-codes: '200,403' -spec: - ingressClassName: alb-currents - rules: - - host: storage.eks.currents-sandbox.work - http: - paths: - - path: / - pathType: Prefix - backend: - service: - name: minio - port: - number: 443 \ No newline at end of file diff --git a/samples/eks/rustfs-eks-ingress.yaml b/samples/eks/rustfs-eks-ingress.yaml new file mode 100644 index 0000000..ed32695 --- /dev/null +++ b/samples/eks/rustfs-eks-ingress.yaml @@ -0,0 +1,29 @@ +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: ingress-rustfs + annotations: + # Set to 'internet-facing' to expose to the public + alb.ingress.kubernetes.io/scheme: internal + alb.ingress.kubernetes.io/group.name: currents + # Set the ARN to a resource managed by AWS Certificate Manager + alb.ingress.kubernetes.io/certificate-arn: "arn:aws:acm:" + alb.ingress.kubernetes.io/target-type: ip + alb.ingress.kubernetes.io/backend-protocol: HTTP + alb.ingress.kubernetes.io/healthcheck-protocol: HTTP + alb.ingress.kubernetes.io/healthcheck-path: /health + alb.ingress.kubernetes.io/success-codes: '200' +spec: + ingressClassName: alb-currents + rules: + # Set the storage DNS name + - host: storage.eks.example.com + http: + paths: + - path: / + pathType: Prefix + backend: + service: + name: rustfs + port: + number: 9000 diff --git a/samples/local/chart-config.yaml b/samples/local/chart-config.yaml index 966ae7b..e90656b 100644 --- a/samples/local/chart-config.yaml +++ b/samples/local/chart-config.yaml @@ -35,11 +35,11 @@ currents: enabled: false host: clickhouse-clickhouse objectStorage: - endpoint: http://minio.localhost - internalEndpoint: https://minio - secretName: currents-minio-user - secretIdKey: CONSOLE_ACCESS_KEY - secretAccessKey: CONSOLE_SECRET_KEY + endpoint: http://rustfs.localhost + internalEndpoint: http://rustfs + secretName: currents-rustfs-user + secretIdKey: RUSTFS_ACCESS_KEY + secretAccessKey: RUSTFS_SECRET_KEY bucket: currents pathStyle: true diff --git a/samples/local/minio-ingress.yaml b/samples/local/rustfs-ingress.yaml similarity index 60% rename from samples/local/minio-ingress.yaml rename to samples/local/rustfs-ingress.yaml index 19cd1bf..479a100 100644 --- a/samples/local/minio-ingress.yaml +++ b/samples/local/rustfs-ingress.yaml @@ -1,12 +1,8 @@ apiVersion: networking.k8s.io/v1 kind: Ingress metadata: - name: ingress-minio + name: ingress-rustfs annotations: - ## Remove if using CA signed certificate - nginx.ingress.kubernetes.io/proxy-ssl-verify: "off" - nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" - nginx.ingress.kubernetes.io/rewrite-target: / nginx.ingress.kubernetes.io/proxy-body-size: "0" nginx.ingress.kubernetes.io/server-snippet: | client_max_body_size 0; @@ -15,13 +11,13 @@ metadata: spec: ingressClassName: currents-nginx rules: - - host: minio.localhost + - host: rustfs.localhost http: paths: - path: / pathType: Prefix backend: service: - name: minio + name: rustfs port: - number: 443 \ No newline at end of file + number: 9000 diff --git a/samples/minio-config.env b/samples/minio-config.env deleted file mode 100644 index 16fc9d1..0000000 --- a/samples/minio-config.env +++ /dev/null @@ -1,2 +0,0 @@ - export MINIO_ROOT_USER="minioRoot" - export MINIO_ROOT_PASSWORD="minioPassword" \ No newline at end of file diff --git a/samples/minio-tenant-helm-config.yaml b/samples/minio-tenant-helm-config.yaml deleted file mode 100644 index 4350798..0000000 --- a/samples/minio-tenant-helm-config.yaml +++ /dev/null @@ -1,127 +0,0 @@ -tenant: - name: currents-minio - configSecret: - name: currents-minio-env-configuration - existingSecret: true - accessKey: null - secretKey: null - pools: - - servers: 1 - ### - # Custom name for the pool - name: pool-0 - ### - # The number of volumes attached per MinIO Tenant Pod / Server. - volumesPerServer: 1 - ### - # The capacity per volume requested per MinIO Tenant Pod. - size: 10Gi - ### - # The `storageClass `__ to associate with volumes generated for this pool. - # - # If using Amazon Elastic Block Store (EBS) CSI driver - # Please make sure to set xfs for "csi.storage.k8s.io/fstype" parameter under StorageClass.parameters. - # Docs: https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/docs/parameters.md - # storageClassName: standard - ### - # Specify `storageAnnotations `__ to associate to PVCs. - storageAnnotations: { } - ### - # Specify `storageLabels `__ to associate to PVCs. - storageLabels: { } - ### - # Specify `annotations `__ to associate to Tenant pods. - annotations: { } - ### - # Specify `labels `__ to associate to Tenant pods. - labels: { } - ### - # - # An array of `Toleration labels `__ to associate to Tenant pods. - # - # These settings determine the distribution of pods across worker nodes. - tolerations: [ ] - ### - # Any `Node Selectors `__ to apply to Tenant pods. - # - # The Kubernetes scheduler uses these selectors to determine which worker nodes onto which it can deploy Tenant pods. - # - # If no worker nodes match the specified selectors, the Tenant deployment will fail. - nodeSelector: { } - ### - # - # The `affinity `__ or anti-affinity settings to apply to Tenant pods. - # - # These settings determine the distribution of pods across worker nodes and can help prevent or allow colocating pods onto the same worker nodes. - affinity: { } - ### - # - # The `Requests or Limits `__ for resources to associate to Tenant pods. - # - # These settings can control the minimum and maximum resources requested for each pod. - # If no worker nodes can meet the specified requests, the Operator may fail to deploy. - resources: { } - ### - # The Kubernetes `SecurityContext `__ to use for deploying Tenant resources. - # - # You may need to modify these values to meet your cluster's security and access settings. - # - # We recommend disabling recursive permission changes by setting ``fsGroupChangePolicy`` to ``OnRootMismatch`` as those operations can be expensive for certain workloads (e.g. large volumes with many small files). - securityContext: - runAsUser: 1000 - runAsGroup: 1000 - fsGroup: 1000 - fsGroupChangePolicy: "OnRootMismatch" - runAsNonRoot: true - ### - # The Kubernetes `SecurityContext `__ to use for deploying Tenant containers. - # You may need to modify these values to meet your cluster's security and access settings. - containerSecurityContext: - runAsUser: 1000 - runAsGroup: 1000 - runAsNonRoot: true - allowPrivilegeEscalation: false - capabilities: - drop: - - ALL - seccompProfile: - type: RuntimeDefault - ### - # - # An array of `Topology Spread Constraints `__ to associate to Operator Console pods. - # - # These settings determine the distribution of pods across worker nodes. - topologySpreadConstraints: [ ] - ### - # - # The name of a custom `Container Runtime `__ to use for the Operator Console pods. - # runtimeClassName: "" - ### - # Array of objects describing one or more buckets to create during tenant provisioning. - # Example: - # - # .. code-block:: yaml - # - # - name: my-minio-bucket - # objectLock: false # optional - # region: us-east-1 # optional - buckets: - - name: currents - ### - # Array of Kubernetes secrets from which the Operator generates MinIO users during tenant provisioning. - # - # Each secret should specify the ``CONSOLE_ACCESS_KEY`` and ``CONSOLE_SECRET_KEY`` as the access key and secret key for that user. - users: - - name: currents-minio-user - - ### - # Directs the Operator to deploy the MinIO S3 API and Console services as LoadBalancer objects. - # - # If the Kubernetes cluster has a configured LoadBalancer, it can attempt to route traffic to those services automatically. - # - # - Specify ``minio: true`` to expose the MinIO S3 API. - # - Specify ``console: true`` to expose the Console. - # - # Both fields default to ``false``. - # exposeServices: - # minio: true \ No newline at end of file diff --git a/samples/rustfs-create-bucket-job.yaml b/samples/rustfs-create-bucket-job.yaml new file mode 100644 index 0000000..b4d1849 --- /dev/null +++ b/samples/rustfs-create-bucket-job.yaml @@ -0,0 +1,29 @@ +apiVersion: batch/v1 +kind: Job +metadata: + name: rustfs-create-bucket +spec: + ttlSecondsAfterFinished: 300 + template: + spec: + restartPolicy: Never + containers: + - name: mc + image: minio/mc:latest + env: + - name: RUSTFS_ACCESS_KEY + valueFrom: + secretKeyRef: + name: currents-rustfs-user + key: RUSTFS_ACCESS_KEY + - name: RUSTFS_SECRET_KEY + valueFrom: + secretKeyRef: + name: currents-rustfs-user + key: RUSTFS_SECRET_KEY + command: + - /bin/sh + - -c + - | + mc alias set rustfs http://rustfs:9000 $RUSTFS_ACCESS_KEY $RUSTFS_SECRET_KEY + mc mb --ignore-existing rustfs/currents diff --git a/samples/rustfs-helm-config.yaml b/samples/rustfs-helm-config.yaml new file mode 100644 index 0000000..cbced67 --- /dev/null +++ b/samples/rustfs-helm-config.yaml @@ -0,0 +1,57 @@ +# RustFS Helm Chart Configuration for Currents Dev/Test +# Standalone mode - single pod for development + +mode: + standalone: + enabled: true + distributed: + enabled: false + +# Use an existing secret for credentials +secret: + existingSecret: "currents-rustfs-user" + +# Service configuration +service: + type: ClusterIP + endpoint: + port: 9000 + console: + port: 9001 + +# Disable gateway API / TraefikService CRD creation +gatewayApi: + gatewayClass: "" + +# Disable built-in ingress (customAnnotations only work with empty ingressClass) +# We create our own ingress for full control +ingress: + enabled: false + +# Storage configuration +storageclass: + name: "" # Uses default storage class + dataStorageSize: "10Gi" + logStorageSize: "256Mi" + +# Resource limits for dev/test +resources: + limits: + cpu: "500m" + memory: "512Mi" + requests: + cpu: "100m" + memory: "128Mi" + +# Pod security context +podSecurityContext: + fsGroup: 10001 + runAsGroup: 10001 + runAsUser: 10001 + +containerSecurityContext: + runAsNonRoot: true + readOnlyRootFilesystem: true + capabilities: + drop: + - ALL