Navigation :

Secure Mode Administration

This guide shows you how to set up and manage Squash’s secure mode.

Introduction

Secure Mode is designed to allow you to apply regular Kubernetes RBAC configurations to Squash debugging activities.
Proper configuration is required for Secure Mode to be secure.
Fortunately, configuration is easy. This guide outlines the best practices for configuring Secure Mode. Ultimately, you are responsible for your cluster’s security, so please apply these recommendations as is appropriate to your use case.

Motivation

Secure Mode is recommended for multi-user clusters.
Reasons to use Secure Mode:
- Prevent users from unexpectedly halting your process in a debug state.
- Prevent malicious exploitation of the Plank pods’ SYS_PTRACE capabilities.
- Prevent malicious exploitation of the Squash pod’s Privileged security context.

Deploy Squash

Quick Start

To “kick the tires” on Squash’s Secure Mode, you can install it with a single command: squashctl deploy squash.
This creates all the resources needed to start using Secure Mode:
- Creates squash-debugger namespace.
- Creates a Service Account with the minimal required permissions.
- Deploys Squash.

Formal Deployment

For shared cluster Squash usage, you should manage your squash deployment through your conventional workflow.
Resources required by Squash include:
- Deployments - Squash
- Service Accounts - Squash, Plank
- Cluster Roles - Squash, Plank
- Cluster Role Bindings - Squash, Plank
For details, see a reference configuration for these resources below.

Configuration requirements for preventing undesired debug activities

Suggestion: do not authorize your users to kubectl exec into the namespace that stores the squash and plank pods.

# Reference: pod exec Policy Rule
# DO NOT enable this permission for the squash-debugger namespace
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: default # DO NOT enable this permission for the squash-debugger namespace
name: pod-exec
rules:
- apiGroups: [""]
resources: ["pods/exec"] # DO NOT enable this permission for the squash-debugger namespace
verbs: ["create"]

Reference configuration

Note: This is a reference configuration for the resources required by Squash.
Note: A priority for our implementation of Secure Mode is minimizing permissions granted to Squash and Plank pods. Our roadmap includes upcoming features that will further reduce the permissions needed by Squash and Plank. These future changes are cited below. We will update these reference configurations as their needs change.

Managed resources

These are the resources you need to configure and manage yourself.

Deployment - Squash

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
  labels:
    app: squash
  name: squash
  namespace: squash-debugger
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: squash
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: squash
    spec:
      containers:
      - env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        - name: HOST_ADDR
          value: $(POD_NAME).$(POD_NAMESPACE)
        - name: NODE_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: spec.nodeName
        image: soloio/squash:dev
        imagePullPolicy: IfNotPresent
        name: squash
        ports:
        - containerPort: 1234
          name: http
          protocol: TCP
        resources: {}
        securityContext:
          privileged: true
          procMount: Default
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /var/run/cri.sock
          name: crisock
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: squash
      serviceAccountName: squash
      terminationGracePeriodSeconds: 30
      volumes:
      - hostPath:
          path: /var/run/dockershim.sock
          type: ""
        name: crisock

Service Account - Squash

apiVersion: v1
kind: ServiceAccount
metadata:
  name: squash
  namespace: squash-debugger

Cluster Role Binding - Squash

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: squash-crb-pods
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: squash-cr-pods
subjects:
- kind: ServiceAccount
  name: squash
  namespace: squash-debugger

Cluster Role - Squash

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: squash-cr-pods
rules:
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - get
  - list
  - watch
  - create
  - delete
- apiGroups:
  - ""
  resources:
  - namespaces
  verbs:
  - list
- apiGroups:
  - squash.solo.io
  resources:
  - debugattachments
  verbs:
  - get
  - list
  - watch
  - create
  - update
  - delete
- apiGroups:
  - rbac.authorization.k8s.io
  resources:
  - clusterrolebindings
  verbs:
  - create
- apiGroups:
  - rbac.authorization.k8s.io
  resources:
  - clusterrole
  verbs:
  - create
- apiGroups:
  - ""
  resources:
# This rule can be removed if you pre-allocate service accounts for Plank
# This rule not be needed in future versions of Squash
# when Squash will expect Plank service accounts already to exist
  - serviceaccount
  verbs:
  - create
- apiGroups:
  - apiextensions.k8s.io
  resources:
  - customresourcedefinitions
  verbs:
  - get
  - list
  - watch
  - create
  - update
  - delete
  - register

Service Account - Plank

apiVersion: v1
kind: ServiceAccount
metadata:
  name: squash-plank
  namespace: squash-debugger

Cluster Role Binding - Plank

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: squash-plank-crb
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: squash-plank-cr
subjects:
- kind: ServiceAccount
  name: squash-plank
  namespace: squash-debugger

Cluster Role - Plank

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: squash-plank-cr
rules:
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - get
  - list
  - watch
  - create
  - delete
- apiGroups:
  - ""
  resources:
  - namespaces
  verbs:
  - list
- apiGroups:
  - squash.solo.io
  resources:
  - debugattachments
  verbs:
  - get
  - list
  - watch
  - create
  - update
  - delete
- apiGroups:
  - apiextensions.k8s.io
  resources:
  - customresourcedefinitions
  verbs:
  - get
  - list
  - watch
  - create
  - update
  - delete

Implicit Resources

These resources are managed by Squash itself.
Note: Squash currently has the ability to create the Service Account, Cluster Role, and Cluster Role Bindings needed by Plank pods. As noted above, this capability will be revoked in the near-term and Squash will expect them to have already been created.

Pod - Plank

Note: Plank pods are created by Squash, you will not need to manage this resource but it is useful to know what it looks like.

In this example, the is debugging a pod named “example-service1-8499d97885” in namespace “my-namespace”

apiVersion: v1
kind: Pod
metadata:
generateName: plank
labels:
debug_attachment_name: example-service1-8499d97885-bgh59
debug_attachment_namespace: my-namespace
squash: plank
name: planklrv4j
namespace: squash-debugger
spec:
containers:
- env:
- name: SQUASH_DEBUG_ATTACHMENT_NAMESPACE
  value: my-namespace
- name: SQUASH_DEBUG_ATTACHMENT_NAME
  value: example-service1-8499d97885-bgh59-example-service1
image: soloio/plank-dlv:dev
imagePullPolicy: IfNotPresent
name: plank
resources: {}
securityContext:
  capabilities:
    add:
    - SYS_PTRACE
  procMount: Default
stdin: true
stdinOnce: true
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
tty: true
volumeMounts:
- mountPath: /var/run/cri.sock
  name: crisock
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
  name: squash-plank-token-jjjxq
  readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
hostPID: true
nodeName: minikube
priority: 0
restartPolicy: Never
schedulerName: default-scheduler
securityContext: {}
serviceAccount: squash-plank
serviceAccountName: squash-plank
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- hostPath:
  path: /var/run/dockershim.sock
  type: ""
name: crisock
- name: squash-plank-token-jjjxq
secret:
  defaultMode: 420
  secretName: squash-plank-token-jjjxq