Align helm chart with best practises

[samuel-lindgren]

The chart currently does not incorporate several recommended best practices such as:

• imagePullSecrets: allowing the image to be hosted and pulled from a private registry
• securityContext: enabling containers run as non-root and follow principle of least privilege
• affinity for controlling pod placement.
• tolerations: for scheduling pods to tolerate specific conditions.
• use gRPC where possible: prefer to use gRPC over TChannel(RPC) due to it being more standardized.

Inspiration
Banzai:

• securityContext
• tolerations
• affinity

Temporal:

• securityContext
• tolerations
• affinity

gRPC:

• One approach could be to use grpc-rpc like Temporal's server-service.yaml
• Another approach could be to use gRPC strictly where possible as shown in below code (from our fork):

{{- range $service := (list "frontend" "matching" "history" "worker") }}
{{- $serviceValues := index $.Values.server $service -}}
apiVersion: v1
kind: Service
metadata:
  name: {{ include "cadence.componentname" (list $ (printf "%s-headless" $service)) }}
  labels: # labels
  annotations: #annotations
spec:
  type: ClusterIP
  clusterIP: None
  publishNotReadyAddresses: true
  ports:
    - port: {{ $serviceValues.service.port }}
      targetPort: rpc
      protocol: TCP
      name: rpc
    {{- /*
       All services but the worker have a gRPC port.
    */ -}}
    {{- if (ne $service "worker") }}
    - port: {{ $serviceValues.service.grpcPort }}
      targetPort: grpc
      protocol: TCP
      name: grpc
    {{- end }}
    {{- if $.Values.metricsEnabled }}
    - name: metrics
      port: {{ $.Values.services.metrics.port }}
      targetPort: metrics
      protocol: TCP
    {{- end }}
  selector:
    app.kubernetes.io/name: {{ include "cadence.name" $ }}
    app.kubernetes.io/instance: {{ $.Release.Name }}
    app.kubernetes.io/component: {{ $service }}

---
{{- end }}