Sunday, September 15, 2019

Testing/Writing Chef Cookbooks

Writing cookbooks

Cookbooks has attributes, recipes, templates etc

Using Community cookbooks
  1. Modify Berksfile in the cookbook by adding cookbook name, say, cookbook 'yum-centos', '~> 3.0.0'
  2. Modify metadata.rb in cookbook by updating dependencies, say, depends 'yum-centos', '~> 3.0.0'
  3. Execute below commands
$ berks install
$ berks upload

Make sure we have below configs/files
  • Chef cookbook
  • Chef Environment
  • Chef Role
Testing cookbook locally

  1. Install vagrant from
  2. Install Virtualbox
  3. Modify .kitchen.yml file referring to community cookbook recipe to test locally.
  4. Goto cookbook directory and execute kitchen commands to build, list, login to new created resource from chef cookbooks.
$ kitchen converge
$ kitchen list
$ kitchen login
Update authentication key
# Connect to chef server 
[pd@ip-disects ~]$ ssh -A -t SSH_SERVER_IP 

# Following commands are executed on chef server 
[pd@ip-disects ~]$ sudo chef-server-ctl user-create praveend Praveen Darshanam Myp@ssw0rd -f /tmp/praveend.key 
ERROR: Conflict 
Response: Username or email address already in use. 

[pd@ip-disects ~]$ sudo chef-server-ctl user-delete praveend 
Do you want to delete the user praveend? (Y/N) y 
Checking organization memberships... 
Checking admin group memberships for 1 org(s). 
FATAL: praveend is in the 'admins' group of the following organization(s): 
- disects

Run this command again with the --remove-from-admin-groups option to remove the user from these admin group(s) automatically. 
[pd@ip-disects ~]$ sudo chef-server-ctl user-delete praveend --remove-from-admin-groups 
Do you want to delete the user praveend? (Y/N) y 
Checking organization memberships... 
Checking admin group memberships for 1 org(s). 
Removing praveend from admins group of 'disects' 
Deleting user praveend. 

[pd@ip-disects ~]$ sudo chef-server-ctl user-create praveend Praveen Darshanam Myp@ssw0rd -f /tmp/praveend.key 
[pd@ip-disects ~]$ sudo chef-server-ctl org-user-add disects praveend --admin 
User praveend is added to admins and billing-admins group

Upload working cookbook to chef server after local testing. Test the cookbook on a cluster node to make sure everything is working fine, this needs some experience though.
$ knife cookbook upload cookbook_name

Kubernetes ingress custom Certificates with valid CA

Irrespective of ingress FQDN, Kubernetes creates Certificates with domain name ingress.local which creates below issues.
CoreOS Dex need certificates from valid CA, self-signed certificates will now work
Gardener dashboard authentication has issues with self-signed certificates. AuthN flow will not happen without accepting invalid Cert error
Accessing ingress in any browser will complain self-signed server error

Fix: Lets encrypt

Install Certbot from LetsEncrypt
$ brew install certbot

Create wildcard Certificate for domain, * 

Before entering Yes to confirm, make sure you add TXT record entry as prompted by certbot.
# create directories named le_wd, le_cd, le_ld before executing below command
$ certbot certonly --manual -d *  --work-dir=le_wd --config-dir=le_cd --logs-dir=le_ld 

# Check if certificates are created
$ certbot certificates --work-dir=le_wd --config-dir=le_cd --logs-dir=le_ld

Certs are located at le_cd/live/ /

Create secret with the Certificates we want to use
$ kubectl create secret tls pd-custom-certs --key --cert -n namespace_of_interest

Configure ingress with the TLS secret.
----SNIP(FQDN 1)---- ingress: enabled: true path: / hosts: - tls: - hosts: - secretName: pd-custom-certs ----SNIP(FQDN 2)---- ingress: enabled: true path: / hosts: - tls: - secretName: pd-custom-certs hosts: -

Accessing ingress should not show invalid Cert errors now.

Kubernetes Pod Security Policies

Start minikube with RBAC and admission-plugins enabled
$ minikube start --extra-config=apiserver.authorization-mode=Node,RBAC --extra-config=apiserver.enable-admission-plugins=PodSecurityPolicy
# or
$ minikube start --extra-config=apiserver.authorization-mode=Node,RBAC --extra-config=apiserver.Admission.PluginNames=PodSecurityPolicy

These commands are not working on my Mac machine, looks like API Server issue as it is not accepting any requests (might not be up).

Create namespace and Service Account
$ kubectl create namespace praveend-psp kubectl create sa test-psp-sa -n praveend-psp

Policy definitions