![\"Bivac](\"https://www.camptocamp.com/wp-content/uploads/bivac-435x400.png\")
Terraform is great for cloud provisioning and has now become a standard tool to deploy infrastructures as code, in a DevOps fashion.
\nMany plugins exist to cover specific needs, from major cloud providers (AWS, GCP, Azure, etc.) to specific app APIs (Grafana, GitHub, or even PostgreSQL). The community provides and maintains additional providers which can be installed and used in any Terraform project as plugins.
\nCamptocamp developed several providers over the last few years. Besides\nthe [official Rancher provider] (https://www.terraform.io/docs/providers/rancher/index.html) which was co-developed by our team and contributed to the community, we maintain providers to integrate Terraform with the PuppetCA, the PuppetDB, as well as the gopass password vault.
\nMore recently, we started having a need to automate FreeIPA resources using Terraform, so we started a new provider.
\nInstalling additional Terraform providers is rather straightforward.\nYou can simply download the binary from the releases page and\ndrop it in your\n~/.terraform.d/plugins\ndirectory.
Like all other Terraform providers, you first need to configure the provider. You can do that using either hardcoded parameters or environment variables. In this second case, we strongly encourage you to make use of summon as a wrapper to dynamically expose the environment variables at call time.
\nprovider freeipa {\n host = "ipa.example.test" # or set $FREEIPA_HOST\n username = "admin" # or set $FREEIPA_USERNAME\n password = "P@S5sw0rd" # or set $FREEIPA_PASSWORD\n insecure = true\n}\n Next, you can start writing resources to manage FreeIPA hosts and DNS records:
\nresource freeipa_host "foo" {\n fqdn = "foo.example.test"\n description = "This is my foo host"\n force = true\n random = true\n userpassword = "abcde"\n}\n\nresource freeipa_dns_record "bar" {\n idnsname = "bar"\n dnszoneidnsname = "myzone"\n dnsttl = 20\n records = ["1.2.3.4"]\n}\n At the moment, this FreeIPA provider only features 2 resource types, to manage FreeIPA hosts and DNS records. Don't hesitate to contribute to it by providing more resource types!
\nThis post was originally published on https://www.camptocamp.com/actualite/automating-freeipa-with-terraform/
\nThis post is also available on DEV.
\n ","pages":[{"url":"/contact/","relativePath":"contact.md","relativeDir":"","base":"contact.md","name":"contact","frontmatter":{"title":"Get in Touch","img_path":"images/contact.jpg","form_id":"contactForm","form_fields":[{"type":"text","name":"name","label":"Name","default_value":"Your name","is_required":true},{"type":"email","name":"email","label":"Email","default_value":"Your email address","is_required":true},{"type":"select","name":"subject","label":"Subject","default_value":"Please select","options":["Error on the site","Sponsorship","Other"]},{"type":"textarea","name":"message","label":"Message","default_value":"Your message"},{"type":"checkbox","name":"consent","label":"I understand that this form is storing my submitted information so I can be contacted."}],"submit_label":"Send Message","template":"contact"},"html":"To get in touch fill the form below.
"},{"url":"/","relativePath":"index.md","relativeDir":"","base":"index.md","name":"index","frontmatter":{"title":"Home","template":"home"},"html":""},{"url":"/posts/a-simple-auth-proxy-for-eks-24dh/","relativePath":"posts/a-simple-auth-proxy-for-eks-24dh.md","relativeDir":"posts","base":"a-simple-auth-proxy-for-eks-24dh.md","name":"a-simple-auth-proxy-for-eks-24dh","frontmatter":{"title":"A Simple Auth Proxy for EKS","stackbit_url_path":"posts/a-simple-auth-proxy-for-eks-24dh","date":"2020-11-11T16:10:48.322Z","excerpt":"How to easily give access to an EKS cluster using an authentication proxy with a PSK","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--bJ8rTXta--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://dev-to-uploads.s3.amazonaws.com/i/49dhgxfjbeqgo2kxo5sr.jpeg","comments_count":0,"positive_reactions_count":8,"tags":["aws","kubernetes","devops","opensource"],"canonical_url":"https://dev.to/camptocamp-ops/a-simple-auth-proxy-for-eks-24dh","template":"post"},"html":"AWS EKS is a great option for a hosted Kubernetes cluster.
\nIt is in particular easy to use for demos and training sessions.
\nHowever, EKS authentication is based off AWS IAM, which means users need an AWS account. Authenticating to EKS typically involves calling the\naws eks get-token\ncommand in your\n.kube/config\nso as to retrieve an authentication token.
As we were setting up EKS for Kubernetes training, we needed a simple way for users without an AWS account to access the cluster, so we created a basic proxy service for the EKS\nget-token\naction.
The proxy can be deployed using Docker, with AWS credentials, e.g.:
\ndocker run --rm -p 8080:8080 \\\n -e AWS_ACCESS_KEY_ID=<AWS_ACCESS_KEY_ID> \\\n -e AWS_SECRET_ACCESS_KEY=<AWS_SECRET_ACCESS_KEY> \\\n -e EKS_CLUSTER_ID=<EKS_CLUSTER_ID> \\\n -e PSK="mysecretstring" \\\n camptocamp/aws-iam-authenticator-proxy:latest\n The rights on the cluster will depend on the user you chose to create the access key.
\nThe PSK is optional, and allows to secure the proxy a little bit.
\nOnce the proxy is started, you can access it at http://localhost:8080?psk=mysecretstring, so you can simply set your\n~/.kube/config\nto use\ncurl\ninstead of\naws\n:
users:\n- name: <cluster_name>\n user:\n exec:\n apiVersion: client.authentication.k8s.io/v1alpha1\n command: curl\n args:\n - -s\n - "http://<your_ip>:8080/?psk=mysecretstring"\n Since you've got an EKS cluster in the first place, you might as well deploy the proxy in it.
\nThe repository provides a Helm chart for that, in the\nk8s\ndirectory of the GitHub project.
You can simply instantiate the chart with the following values:
\neks_cluster_id: "<EKS_CLUSTER_ID>"\npsk: "mysecretstring"\naws:\n access_key_id: "<AWS_ACCESS_KEY_ID>"\n secret_access_key: "<AWS_SECRET_ACCESS_KEY>"\n The AWS credentials will be stored in a Kubernetes secret and passed to the container.
\nHowever, since we're in AWS, we can also use IAM roles for service accounts and bypass the access keys altogether. This is a much cleaner approach.
\nHere's how to do it, using Terraform to create the role and deploy the proxy.
\nFirst, create a role linked to OIDC:
\nmodule "iam_assumable_role_aws_iam_authenticator_proxy" {\n source = "terraform-aws-modules/iam/aws//modules/iam-assumable-role-with-oidc"\n version = "3.3.0"\n create_role = true\n number_of_role_policy_arns = 0\n role_name = "aws-iam-authenticator-proxy"\n provider_url = replace(module.cluster.cluster_oidc_issuer_url, "https://", "")\n oidc_fully_qualified_subjects = ["system:serviceaccount:yournamespace:aws-iam-authenticator-proxy"]\n}\n replacing\nyournamespace\nwith the Kubernetes namespace where you will be deploying the proxy.
Now we can configure the cluster to use map that role to the Kubernetes role we want (e.g.\nsystem::masters\nto make it cluster admin).
We'll create a random PSK and generate the Kubeconfig file to use\ncurl\nwith the proxy:
data "aws_vpc" "this" {\n id = var.vpc_id\n}\n\ndata "aws_subnet_ids" "private" {\n vpc_id = data.aws_vpc.this.id\n\n tags = {\n "kubernetes.io/role/internal-elb" = "1"\n }\n}\n\nmodule "cluster" {\n source = "terraform-aws-modules/eks/aws"\n version = "13.1.0"\n\n cluster_name = var.cluster_name\n cluster_version = "1.18"\n\n subnets = data.aws_subnet_ids.private.ids\n vpc_id = var.vpc_id\n enable_irsa = true\n map_roles = [\n {\n rolearn = module.iam_assumable_role_aws_iam_authenticator_proxy.this_iam_role_arn,\n username = module.iam_assumable_role_aws_iam_authenticator_proxy.this_iam_role_name,\n groups = ["system:masters"]\n },\n ]\n\n worker_groups = [\n {\n instance_type = "m5a.large"\n asg_desired_capacity = 2\n asg_max_size = 3\n }\n ]\n\n kubeconfig_aws_authenticator_command = "curl"\n kubeconfig_aws_authenticator_command_args\t= [\n "-s",\n "https://${var.auth_url}/?psk=${random_password.auth_proxy_psk.result}",\n ]\n}\n\nresource "random_password" "auth_proxy_psk" {\n length = 16\n special = false\n}\n Finally, we can deploy the proxy in Kubernetes using Helm:
\nresource "helm_release" "aws-iam-authenticator-proxy" {\n name = "aws-iam-authenticator-proxy"\n chart = "https://github.com/camptocamp/aws-iam-authenticator-proxy/tree/master/k8s"\n namespace = "aws-iam-authenticator-proxy"\n dependency_update = true\n create_namespace = tr\n\n values = [\n << EOT\neks_cluster_id: "${var.cluster_name}"\npsk: "${random_password.auth_proxy_psk.result}"\nserviceAccount:\n name: "aws-iam-authenticator-proxy"\n annotations:\n eks.amazonaws.com/role-arn: ${module.iam_assumable_role_aws_iam_authenticator_proxy.this_iam_role_arn}\nEOT\n ]\n\n depends_on = [\n module.cluster,\n ]\n}\n You can add an\nIngress\nor configure the\nService\nto use an L4\nLoadBalancer\nby tuning the Helm values.
This post is also available on DEV.
\n "},{"url":"/posts/add-puppet-tag-142l/","relativePath":"posts/add-puppet-tag-142l.md","relativeDir":"posts","base":"add-puppet-tag-142l.md","name":"add-puppet-tag-142l","frontmatter":{"title":"Add #puppet tag","stackbit_url_path":"posts/add-puppet-tag-142l","date":"2020-06-12T06:08:13.053Z","excerpt":"It would be great to attract more DevOps-related content to dev.to. With a few other people, I've sta...","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--X2kztkXc--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://www.camptocamp.com/wp-content/uploads/xformations_puppet1-720x400.png.pagespeed.ic.UU2oY1Zlj8.webp","comments_count":3,"positive_reactions_count":2,"tags":["meta","puppet","puppetize","discuss"],"canonical_url":"https://dev.to/raphink/add-puppet-tag-142l","template":"post"},"html":"It would be great to attract more DevOps-related content to dev.to. With a few other people, I've started blogging about Puppet and would really appreciate if it could become an official tag!
\nThis post is also available on DEV.
\n "},{"url":"/posts/automatic-renewal-of-puppet-certificates-28pm/","relativePath":"posts/automatic-renewal-of-puppet-certificates-28pm.md","relativeDir":"posts","base":"automatic-renewal-of-puppet-certificates-28pm.md","name":"automatic-renewal-of-puppet-certificates-28pm","frontmatter":{"title":"Automatic Renewal of Puppet Certificates","stackbit_url_path":"posts/automatic-renewal-of-puppet-certificates-28pm","date":"2020-05-04T06:36:04.499Z","excerpt":"Everyone who has been using Puppet with a self-signed CA for more than 5 years knows that dreaded time: the time when the CA must be renewed.","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--X2kztkXc--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://www.camptocamp.com/wp-content/uploads/xformations_puppet1-720x400.png.pagespeed.ic.UU2oY1Zlj8.webp","comments_count":0,"positive_reactions_count":7,"tags":["puppet","devops","opensource","security"],"canonical_url":"https://www.camptocamp.com/actualite/automatic-renewal-of-puppet-certificates/","template":"post"},"html":"Everyone who has been using Puppet with a self-signed CA for over 5 years knows the dreaded time: the time when the CA must be renewed.
\nThe traditional approach is to create a new CA, and then use another mean to renew the certificates for all the nodes (SSH, MCollective, Ansible, etc.).
\nAnother possibility is to keep the same CA keys and generate a new CA certificate. There is actually an official module to do that. This module allows you to easily revive a CA that is about to expire in such a way that the new CA certificate is valid for current node certificates. As a result, you don't need to renew the node certificates right away; you only need to distribute the new CA certificate to ensure the cached version does not expire!
\nThere is however a consequence: node certificates will start to expire. Nodes over 5 years old will expire as soon as the CA expires. And so they need to be renewed, too.
\nWhat if the Puppet agent itself could ensure its certificate is always\nvalid? This can be achieved using two things:
\npuppet_certificate\nresource type.The former is a standard Puppet feature, with a simple principle: embark a secret in the Puppet CSR, which will be checked by a script on the CA.
\nThis approach allows to easily automate node provisioning by making nodes automatically register into Puppet.
\nThe simplest form uses a shared password in the\ncsr_attributes.yaml\nfile on the Puppet node.
Once you have put together a way to autosign certificates, let's see how to automatically renew these certificates. We'll use the puppet_certificate Puppet module for that. Here is the kind of code you could use:
\nclass profile::puppet::certificate (\n String $psk,\n) {\n file { '/etc/puppetlabs/puppet/csr_attributes.yaml':\n ensure => file,\n owner => 'root',\n group => 'root',\n mode => '0440',\n content => "---\\ncustom_attributes:\\n 1.2.840.113549.1.9.7: '${psk}'\\n",\n }\n ~> puppet_certificate { $::trusted['certname']:\n ensure => valid,\n onrefresh => 'regenerate',\n waitforcert => 60,\n renewal_grace_period => 20,\n clean => true,\n }\n}\n This code will:
\ncsr_attributes.yaml\nfile to inject the psk into it;In addition:
\nensure => valid\n);Note that this only works if the certificate is cleaned from the Puppet CA before it gets regenerated. This is the point of the\nclean => true\nattribute. By default, however, the Puppet CA does not accept remote cleaning of certificates. You can allow nodes to clean their own certificates (and no other) by adding this to your Puppetserver's\nauth.conf\nfile:
{\n name: "Allow nodes to delete their own certificates",\n match-request: {\n path: "^/puppet-ca/v1/certificate(_status|_request)?/([^/]+)$",\n type: regex,\n method: [delete]\n },\n allow: "$2",\n sort-order: 500\n}\n While it is possible to use a simple shared password in the\ncsr_attributes.yaml\nfor autosigning, it means all your nodes will contain that psk, which is valid to create new certificates on the Puppet CA. This is not very secure, and this can be improved by using unique tokens for each node, so that a token can only be used to generate a certificate for a specific node.
You could achieve this with tools such as Vault. Another idea is to generate a composite secret on the Puppet master by mixing the psk with the certname and possibly any certificate extension you want to enforce. You then need to adapt your autosign policy script to generate the same composite secret, which ensures that each node can only generate its own certificate, without changing its extensions.
\nThis post was originally published on camptocamp.com
\nThis post is also available on DEV.
\n "},{"url":"/posts/automating-freeipa-with-terraform-43b7/","relativePath":"posts/automating-freeipa-with-terraform-43b7.md","relativeDir":"posts","base":"automating-freeipa-with-terraform-43b7.md","name":"automating-freeipa-with-terraform-43b7","frontmatter":{"title":"Automating FreeIPA with Terraform","stackbit_url_path":"posts/automating-freeipa-with-terraform-43b7","date":"2020-04-30T10:48:36.114Z","excerpt":"The FreeIPA Terraform provider allows to automate creation and management of FreeIPA resources.","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--EFbU1JTl--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://www.camptocamp.com/wp-content/uploads/xterraform_bandeau.png.pagespeed.ic.neAGqH-_lX.webp","comments_count":0,"positive_reactions_count":0,"tags":["terraform","devops","freeipa","opensource"],"canonical_url":"https://www.camptocamp.com/actualite/automating-freeipa-with-terraform/","template":"post"},"html":"Terraform is great for cloud provisioning and has now become a standard tool to deploy infrastructures as code, in a DevOps fashion.
\nMany plugins exist to cover specific needs, from major cloud providers (AWS, GCP, Azure, etc.) to specific app APIs (Grafana, GitHub, or even PostgreSQL). The community provides and maintains additional providers which can be installed and used in any Terraform project as plugins.
\nCamptocamp developed several providers over the last few years. Besides\nthe [official Rancher provider] (https://www.terraform.io/docs/providers/rancher/index.html) which was co-developed by our team and contributed to the community, we maintain providers to integrate Terraform with the PuppetCA, the PuppetDB, as well as the gopass password vault.
\nMore recently, we started having a need to automate FreeIPA resources using Terraform, so we started a new provider.
\nInstalling additional Terraform providers is rather straightforward.\nYou can simply download the binary from the releases page and\ndrop it in your\n~/.terraform.d/plugins\ndirectory.
Like all other Terraform providers, you first need to configure the provider. You can do that using either hardcoded parameters or environment variables. In this second case, we strongly encourage you to make use of summon as a wrapper to dynamically expose the environment variables at call time.
\nprovider freeipa {\n host = "ipa.example.test" # or set $FREEIPA_HOST\n username = "admin" # or set $FREEIPA_USERNAME\n password = "P@S5sw0rd" # or set $FREEIPA_PASSWORD\n insecure = true\n}\n Next, you can start writing resources to manage FreeIPA hosts and DNS records:
\nresource freeipa_host "foo" {\n fqdn = "foo.example.test"\n description = "This is my foo host"\n force = true\n random = true\n userpassword = "abcde"\n}\n\nresource freeipa_dns_record "bar" {\n idnsname = "bar"\n dnszoneidnsname = "myzone"\n dnsttl = 20\n records = ["1.2.3.4"]\n}\n At the moment, this FreeIPA provider only features 2 resource types, to manage FreeIPA hosts and DNS records. Don't hesitate to contribute to it by providing more resource types!
\nThis post was originally published on https://www.camptocamp.com/actualite/automating-freeipa-with-terraform/
\nThis post is also available on DEV.
\n "},{"url":"/posts/backup-your-container-data-2f3f/","relativePath":"posts/backup-your-container-data-2f3f.md","relativeDir":"posts","base":"backup-your-container-data-2f3f.md","name":"backup-your-container-data-2f3f","frontmatter":{"title":"Backup your Container Data","stackbit_url_path":"posts/backup-your-container-data-2f3f","date":"2020-04-30T10:48:01.446Z","excerpt":"Containers have become a great facility to easily deploy applications, whether locally or on orchestrated clusters. However, containers are ephemeral, meaning their data should be stored externally and should be backed up.","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--Qh3lKBaH--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://www.camptocamp.com/wp-content/uploads/banner.png","comments_count":0,"positive_reactions_count":5,"tags":["opensource","backup","showdev","kubernetes"],"canonical_url":"https://www.camptocamp.com/actualite/backup-your-container-data/","template":"post"},"html":"Containers have become a great facility to easily deploy applications, whether locally or on orchestrated clusters.
\nHowever, containers are ephemeral, meaning their data should be stored externally. When possible, they can be stored using databases or object storage. Most often though, you will need to resort to using data volumes, mounted inside your containers. How then can be perform a backup of this data?
\nContrarily to the traditional situation in application deployment, the location of critical data in containers is known, since it uses named volumes. We can thus connect to the Docker socket or the API managing the volumes to list them and perform the backups.
\nBivac is a tool created to do just that. It can be plugged to either a Docker socket, a Rancher API, or a Kubernetes server. It will then list the volumes on the platform and automatically back them up on a regular basis, using Restic to transfer the data to an object storage provider (e.g. AWS S3).
\n
In addition, Bivac can provide metrics on the backup statuses as it exposes a Prometheus endpoint.
\nUsing the REST client, backups can be listed, executed on demand, and it is also possible to restore volumes.
\nBivac can easily be installed a binary or a container. Here are some examples, deploying it locally on Docker, or using Kubernetes.
\nThe following\ndocker-compose.yml\nfile can be used to deploy the Bivac manager:
---\nversion: '3'\nservices:\n bivac:\n image: camptocamp/bivac:2.2\n command: "manager -v"\n ports:\n - "8182:8182"\n volumes:\n - "/var/run/docker.sock:/var/run/docker.sock:ro"\n environment:\n BIVAC_AGENT_IMAGE: camptocamp/bivac:2.1\n BIVAC_SERVER_PSK: super-secret-psk\n RESTIC_PASSWORD: not-so-good-password\n BIVAC_TARGET_URL: s3:my-bucket\n AWS_ACCESS_KEY_ID: XXXXX\n AWS_SECRET_ACCESS_KEY: XXXXX\n Additionally, you can also deploy a local Prometheus server to retrieve the metrics. See the full example.
\nThe easiest way to deploy a Bivac manager on Kubernetes is to use Camptocamp's Helm chart:
\n$ helm repo add camptocamp http://charts.camptocamp.com\n$ helm install camptocamp/bivac --version 1.0.0\n The CLI can be downloaded from the releases page. Once the binary is installed, you can use it to list backups, perform backups, or restore data.
\nThe CLI needs to be connected to the Bivac manager, using its HTTP URL and PSK (defined in the deployment). This can be performed using either the\n--remote.address\nand\n--server.psk\noptions, or by setting the\nBIVAC_REMOTE_ADDRESS\nand\nBIVAC_SERVER_PSK\n.
The\nbivac volumes\ncommand lets you list the volumes managed by Bivac:
While Bivac automatically performs backups at a regular interval, the CLI can also be used to trigger backups manually:
\n\nBivac stores restic backups on object storage and lets you restore them using the\nbackup restore\ncommand:
More features are available, such as the ability to manage a remote Restic repository. See the documentation for more information.
\nBivac allows to easily backup data, monitor their status and restore them, whether you are using raw Docker, Rancher volumes or Kubernetes.
\nThis post was originally published on https://www.camptocamp.com/actualite/backup-your-container-data/
\nThis post is also available on DEV.
\n "},{"url":"/posts/bitten-by-ha-puppetdb-postgresql-1eld/","relativePath":"posts/bitten-by-ha-puppetdb-postgresql-1eld.md","relativeDir":"posts","base":"bitten-by-ha-puppetdb-postgresql-1eld.md","name":"bitten-by-ha-puppetdb-postgresql-1eld","frontmatter":{"title":"Bitten by HA: PuppetDB & PostgreSQL","stackbit_url_path":"posts/bitten-by-ha-puppetdb-postgresql-1eld","date":"2020-05-22T14:32:17.632Z","excerpt":"When PuppetDB started misbehaving, it took us quite a while to realize the problem was somewhere else…","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--mZqZZYCG--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://dev-to-uploads.s3.amazonaws.com/i/ux2ltllgi7mjem72mf2z.png","comments_count":0,"positive_reactions_count":5,"tags":["puppet","postgres","ha","debugging"],"canonical_url":"https://dev.to/camptocamp-ops/bitten-by-ha-puppetdb-postgresql-1eld","template":"post"},"html":"Last Wednesday morning, a colleague informed me that our internal Puppet infrastructure was performing slowly. Looking at the Puppetboard, I quickly realized there was another issue: all the nodes were marked as unreported, with the last report dating from more than 24 hours in the past.
\nI checked the PuppetDB logs and saw that the reports were coming fine and being saved, so something else was wrong.
\nAfter a few hours of debugging, I still had no clue so I resorted to the option of upgrading the PuppetDB. I ideally wanted to stay with PuppetDB 5.x to avoid a major upgrade, but there was no available official Docker image for PuppetDB 5.x above 5.2.7 (which we are using).
\nSo I upgraded to PuppetDB 6.9.1 and then PuppetDB 6.10.1.
\nAt first, the connection with PostgreSQL failed as PuppetDB now defaults to fully verifying the PostgreSQL SSL certificate against the Puppet CA. So I had to modify the PostgreSQL connection string to include\n?sslmode=require\nand restart the PuppetDB.
The logs seemed fine at first, but the Puppetboard kept returning an error 500. That's when I realized that PuppetDB now required an access file to allow hosts by IP range. I modified our Helm chart to include a new configuration file, but Puppetboard still wasn't happy.
\nI restarted the PuppetDB and started seeing weird log lines looping infinitely, right after the migration step in startup:
\n2020-05-20 18:55:40,169 WARN [clojure-agent-send-off-pool-1] [p.p.jdbc] Caught exception. Last attempt, throwing exception.\n At this point, Charlie Sharpsteen started helping me to debug the issue.
\nI tried manual\ncurl\nrequests to the PuppetDB and the logs started sprouting stack traces mentioning both JDBC connection issues as well as the database schema version being wrong:
...\nCaused by: java.sql.SQLTransientConnectionException: PDBReadPool - Connection is not available, request timed out after 3000ms.\n...\nCaused by: com.zaxxer.hikari.pool.PoolBase$ConnectionSetupException: org.postgresql.util.PSQLException: ERROR: Please run PuppetDB with the migrate option set to true\n to upgrade your database. The detected migration level 66 is\n out of date.\n...\n I connected to the PostgreSQL database and checked it. Everything looked fine, and\nselect max(version) from schema_migrations;\nreturned\n74\nas expected, not\n66\n. So where did this number come from? Charlie started suspecting that there was another database involved…
Totally out of other options, I decided to remove the lines with versions above 66 in the\nschema_migrations\ntable and see if restarting the PuppetDB would finalize the migration. That was a huge failure, as the migration scripts are not idempotent, as could be expected.
I was left with only one option: dropping the database and restoring it.\nBut then PostgreSQL refused to drop the database, saying it was read-only. I tried forcing read-write, but the database was marked in recovery.
\nThat's when I gave up for the day (as it was already past 23:00). I turned off the PuppetDB service entirely (scaled the deployment to 0 replicas actually) and went to bed, letting the nodes apply catalogs from cache for the next 30 hours (since Thursday was off).
\nThis morning, we got back to debugging this problem, and things started making more sense.
\nFirst off, it turned out I was trying to drop the database on a slave cluster. I had ended up on the slave by using a production CNAME DNS entry which pointed to both the master and slave in round-robin…
\nOnce my colleague Julien had helped me realize that, he was able to drop the database on the master. We restarted the PuppetDB in version 6.10.1. But the errors were still there…
\nWe rolled back to PuppetDB 5.2.7 and a clean database… Everything started fine, but the Puppetboard still showed all nodes as unreported! Where could it get these nodes if the database had been wiped‽
\nThis led us to the conclusion that the data was still somewhere else… on the slave…
\nSo here's what happened.
\nEarlier this week, there was an outage with the object storage facility we're using for wal-g, our PostgreSQL backup tool.
\n![\"PostgreSQL](\"https://dev-to-uploads.s3.amazonaws.com/i/ux2ltllgi7mjem72mf2z.png\")
This led to a disk full on our master PostgreSQL machine. The disk full was very short as PostgreSQL restarted and removed all its WALs so it went unnoticed. However, this also broke replication, so the slave PostgreSQL database ended up stuck on that date. For some reason, we missed the replication alert.
\n![\"Wal](\"https://dev-to-uploads.s3.amazonaws.com/i/ayytz0tufaf6mrh83q29.png\")
PuppetDB is configured to write to master and read from slave. This is why all our nodes were unreported in Puppetboard (since they came from slave), even though PuppetDB kept writing the reports properly (in master)! This also explains the weird errors after upgrading to PuppetDB 6, since migration was properly done on the master (to schema v74) but read requests went to the slave (stuck in schema v66).
\nSince we had wiped the master's database this morning, we ended up restoring from the slave's version, and going back to PuppetDB 5.2.7 until we can properly solve the Jolokia potential issues with external access to the PuppetDB API.
\nAll nodes in Puppetboard have now returned to normal.
\nThis post is also available on DEV.
\n "},{"url":"/posts/colored-wrappers-for-kubectl-2pj1/","relativePath":"posts/colored-wrappers-for-kubectl-2pj1.md","relativeDir":"posts","base":"colored-wrappers-for-kubectl-2pj1.md","name":"colored-wrappers-for-kubectl-2pj1","frontmatter":{"title":"Colored wrappers for kubectl","stackbit_url_path":"posts/colored-wrappers-for-kubectl-2pj1","date":"2020-10-06T19:50:58.078Z","excerpt":"Kubectl commands, but in color","thumb_img_path":null,"comments_count":1,"positive_reactions_count":11,"tags":["kubernetes","devops","cli","zsh"],"canonical_url":"https://dev.to/raphink/colored-wrappers-for-kubectl-2pj1","template":"post"},"html":"When using Kubernetes,\nkubectl\nis the command we use the most to visualize and debug objects.
However, it currently does not support colored output, though there is a feature request opened for this.
\nLet's see how we can add color support. I'll be using zsh with oh my zsh.
\nEdit: this feature was merged in oh my zsh, so it is now standard.
\nLet's make this extension into a zsh plugin called\nkubectl_color\n:
❯ mkdir -p ~/.oh-my-zsh/custom/plugins/kubectl_color\n❯ touch ~/.oh-my-zsh/custom/plugins/kubectl_color/kubectl_color.plugin.zsh\n Now we need to fill in this plugin.
\nLet's start with JSON, by adding an alias that colorizes JSON output using the infamous \njq\n:
kj() {\n kubectl "$@" -o json | jq\n}\n\ncompdef kj=kubectl\n The\ncompdef\nline ensures the\nkj\nfunction gets autocompleted just like\nkubectl\n.
Edit: I've added another wrapper for \nfx\n, which provides a dynamic way to parse JSON:
kjx() {\n kubectl "$@" -o json | fx\n}\n\ncompdef kjx=kubectl\n Just like for JSON, we can use \nyh\n to colorize YAML output:
ky() {\n kubectl "$@" -o yaml | yh\n}\n\ncompdef ky=kubectl\n Our plugin is now ready, we only need to activate it in\n~/.zshrc\nby adding it to the list of plugins, e.g.:
plugins=(git ruby kubectl kubectl_color)\n and with\nfx\n:
This post is also available on DEV.
\n "},{"url":"/posts/decomissioning-with-puppet-report-purge-unmanaged-resources-1jgk/","relativePath":"posts/decomissioning-with-puppet-report-purge-unmanaged-resources-1jgk.md","relativeDir":"posts","base":"decomissioning-with-puppet-report-purge-unmanaged-resources-1jgk.md","name":"decomissioning-with-puppet-report-purge-unmanaged-resources-1jgk","frontmatter":{"title":"Decomissioning with Puppet: report & purge unmanaged resources","stackbit_url_path":"posts/decomissioning-with-puppet-report-purge-unmanaged-resources-1jgk","date":"2020-07-23T14:42:09.777Z","excerpt":"Puppet can let you purge resources you do not manage explicitely","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--X2kztkXc--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://www.camptocamp.com/wp-content/uploads/xformations_puppet1-720x400.png.pagespeed.ic.UU2oY1Zlj8.webp","comments_count":0,"positive_reactions_count":4,"tags":["puppet","devops","cfgmgmt","tutorial"],"canonical_url":"https://dev.to/camptocamp-ops/decomissioning-with-puppet-report-purge-unmanaged-resources-1jgk","template":"post"},"html":"Puppet lets you manage resources explicitely. But did you know you can also dynamically purge unmanaged resources using Puppet?
\nA user in your organization just left, and you need to remove their account from all nodes. If you were managing their account with Puppet —whether with a\nuser\nresource type or using an accounts module—, you need to make sure this user is absent:
user { 'jdoe':\n ensure => absent,\n}\n Great. Job done. Now, how long should this resource be kept in your code? One hour? One week? One year? What if an old node that was turned off wakes up months from now with this account activated?
\nTo be honest, if a node turned off for months suddenly wakes up, you'll probably have more issues than just old users if your Puppet code base is quite active…\nHowever, purging all unknown users would be a much easier approach than managing them explicitely!
\nAs explained in a previous post about managing files in Puppet, Puppet has the ability of purging unmanaged resources. I'll let you see the post for more explanations on how this works:
\n\nWhat if instead of purging, I'd just like Puppet to report the unmanaged resources but not do anything about them?
\nLuckily for us,\nnoop\nworks fine with the\npurge\ntype, so you can use something like:
purge { 'user':\n noop => true,\n unless => [\n ['uid', '<', '1000'],\n ['name', '==', 'nobody'],\n ],\n}\n This code will mark all users with a UID above 999 (except the\nnobody\nuser) to be purged, but it won't do it. As a result, you'll get\nnoop\nresources in your reports, for example in Puppetboard:
![\"Noop](\"https://dev-to-uploads.s3.amazonaws.com/i/3xu0q5i3e98bv27ih117.png\")
And then in the report, you'll see the unmanaged users:
\n![\"Report](\"https://dev-to-uploads.s3.amazonaws.com/i/ckpsim9bx6igwp09it4l.png\")
If you see users that should be purged, you can add again a\nuser\nresource in your Puppet code to ensure their absence:
user { 'iperf':\n ensure => absent,\n}\n Another option is to make it a bit more dynamic. I've added an option in my\naccounts\nbase class to use a dynamic fact to purge users on demand:
class osbase::accounts (\n Boolean $purge_users = str2bool($facts['purge_users']),\n) {\n purge { 'user':\n noop => !$purge_users,\n unless => [\n ['uid', '<', '1000'],\n ['name', '==', 'nobody'],\n ],\n }\n}\n The\npurge_users\nfact doesn't exist by default, so I can define it on the go when I need to purge users.\nNow I can run\npuppet apply\non a node and force purging the users with:
$ FACTER_purge_users=y puppet agent -t\n And all unmanaged users will be removed from the node!
\nDo you have specific Puppet needs? Contact us, we can help you!
\nThis post is also available on DEV.
\n "},{"url":"/posts/deploying-public-keys-in-docker-containers-41cd/","relativePath":"posts/deploying-public-keys-in-docker-containers-41cd.md","relativeDir":"posts","base":"deploying-public-keys-in-docker-containers-41cd.md","name":"deploying-public-keys-in-docker-containers-41cd","frontmatter":{"title":"Deploying public keys in Docker containers","stackbit_url_path":"posts/deploying-public-keys-in-docker-containers-41cd","date":"2020-05-08T07:57:33.416Z","excerpt":"One of the hard problems to solve when using Docker in production is deploying secrets. githut_pki makes SSH key deployment easy.","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--jI5YGum6--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://dev-to-uploads.s3.amazonaws.com/i/1doni1qp0l9lqk240myf.png","comments_count":0,"positive_reactions_count":6,"tags":["devops","showdev","opensource","docker"],"canonical_url":"https://www.camptocamp.com/en/actualite/deploying-public-keys-in-docker-containers/","template":"post"},"html":"One of the hard problems to solve when using Docker in production is deploying secrets. In particular, public keys are hard to deploy because they are multiline and there is usually one key per authorized user.
\nSince all our users have accounts on GitHub with their SSH key, it made sense to us to use GitHub as a centralized PKI for SSH keys. Starting with a simple Ruby script connecting to the GitHub API, we soon realized we would need a generic way of deploying public keys from GitHub if we persisted in this approach.
\nThis gave birth to the github_pki, a generic command line tool using the GitHub API to deploy SSH and X509 keys from GitHub organizations, teams, and individual users.
\nInstalling can be done from source:
\nFROM debian:jessie\n\nENV GOPATH=/go\nRUN apt-get update && apt-get install -y golang-go git \\\n && go get github.com/camptocamp/github_pki \\\n && apt-get autoremove -y golang-go git \\\n && rm -rf /var/lib/apt/lists/*\n Or by inheriting one of the official Docker images.
\nThe github_pki command can then simply be called from within an entrypoint script to deploy keys:
\n# !/bin/sh\n\n# Deploy users keys as X509 public keys to SSL_DIR\nSSL_DIR=/etc/puppetlabs/mcollective/clients /go/bin/github_pki\n\n# Deploy user keys as an authorized_keys file\nAUTHORIZED_KEYS=/root/.ssh/authorized_keys /go/bin/github_pki\n Various environment variables can be used to tune which keys should be deployed:
\n$ docker run -e AUTHORIZED_KEYS=/root/.ssh/authorized_keys \\\n -e SSL_DIR=/etc/test/ssl \\\n -e GITHUB_ORG="myorg" \\\n -e GITHUB_TEAM="mypals" \\\n -e GITHUB_USERS="otheruser" \\\n -e GITHUB_TOKEN=398d6d326a546d40f3f1ef93345d1fc5ee0f0j38 \\\n mydockerimage\nrun-parts: executing /docker-entrypoint.d/25-populate-ssl-clients.sh\ntime="2016-03-22T09:45:52Z" level=info msg="Adding users for team mypals" \ntime="2016-03-22T09:45:52Z" level=info msg="Adding user bob" \ntime="2016-03-22T09:45:52Z" level=info msg="Adding user alice" \ntime="2016-03-22T09:45:52Z" level=info msg="Adding individual user otheruser" \ntime="2016-03-22T09:45:53Z" level=info msg="Getting keys for user bob" \ntime="2016-03-22T09:45:53Z" level=info msg="Getting keys for user alice" \ntime="2016-03-22T09:45:53Z" level=info msg="Getting keys for user otheruser"\ntime="2016-03-22T09:45:59Z" level=info msg="Generating /root/.ssh/authorized_keys" \ntime="2016-03-22T09:45:59Z" level=info msg="Dumping X509 keys to /etc/puppetlabs/mcollective/clients" \ntime="2016-03-22T09:45:59Z" level=info msg="Converting key bob/1325852 to X509" \ntime="2016-03-22T09:45:59Z" level=info msg="Converting key alice/123756 to X509" \ntime="2016-03-22T09:45:59Z" level=info msg="Converting key alice/7845928 to X509" \ntime="2016-03-22T09:45:59Z" level=info msg="Converting key otheruser/8540586 to X509"\n This blog post was originally published on camptocamp.com
\nThis post is also available on DEV.
\n "},{"url":"/posts/error-console-highlighting-1c60/","relativePath":"posts/error-console-highlighting-1c60.md","relativeDir":"posts","base":"error-console-highlighting-1c60.md","name":"error-console-highlighting-1c60","frontmatter":{"title":"Error console highlighting","stackbit_url_path":"posts/error-console-highlighting-1c60","date":"2020-05-13T20:47:24.431Z","excerpt":"When composing posts on dev.to, is there a way to dispensary display error messages so they appear as...","thumb_img_path":null,"comments_count":3,"positive_reactions_count":6,"tags":["discuss","question","writing","markdown"],"canonical_url":"https://dev.to/raphink/error-console-highlighting-1c60","template":"post"},"html":"When composing posts on dev.to, is there a way to dispensary display error messages so they appear as console output, but in red?
\nI've tried things like:
\n\n\n\n```error\nMy error message\n```\n\n\n\nor
\n<pre>\n<code style="color:red;font-weight:bold">\nMy error message\n</code>\n</pre>\n to no avail…
\nIs there a way to achieve this?
\nThis post is also available on DEV.
\n "},{"url":"/posts/generic-blog-17ni/","relativePath":"posts/generic-blog-17ni.md","relativeDir":"posts","base":"generic-blog-17ni.md","name":"generic-blog-17ni","frontmatter":{"title":"dev.to as a generic blog?","stackbit_url_path":"posts/generic-blog-17ni","date":"2020-05-02T18:41:25.829Z","excerpt":"Is it a good idea to use dev.to for other subjects than development?","thumb_img_path":null,"comments_count":5,"positive_reactions_count":5,"tags":["discuss","beginners"],"canonical_url":"https://dev.to/raphink/generic-blog-17ni","template":"post"},"html":"Do any of you use dev.to as a generic blog, besides development-related subjects (I'm thinking genealogy for example)?
\nWhat would be the pros and cons?
\nThis post is also available on DEV.
\n "},{"url":"/posts/getting-puppet-report-metrics-from-puppetdb-6bp/","relativePath":"posts/getting-puppet-report-metrics-from-puppetdb-6bp.md","relativeDir":"posts","base":"getting-puppet-report-metrics-from-puppetdb-6bp.md","name":"getting-puppet-report-metrics-from-puppetdb-6bp","frontmatter":{"title":"Getting Puppet Report Metrics from PuppetDB","stackbit_url_path":"posts/getting-puppet-report-metrics-from-puppetdb-6bp","date":"2020-06-02T09:22:19.303Z","excerpt":"Instead of sending metrics from the Puppetserver to Prometheus, they can be retrieved using the PuppetDB Metrics API.","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--DX3k0Pdb--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://dev-to-uploads.s3.amazonaws.com/i/jczukvqf1fs7dau9jnfp.png","comments_count":0,"positive_reactions_count":4,"tags":["puppet","showdev","opensource","devops"],"canonical_url":"https://dev.to/camptocamp-ops/getting-puppet-report-metrics-from-puppetdb-6bp","template":"post"},"html":"Puppet agent run reports contain useful metrics, such as the number of resources that were modified or failed to apply, or how much time each step of the run took.
\nThe traditional way of retrieving these metrics is using a report processor on the Puppet master.
\nSince Prometheus is now a de facto standard in metrics collection, there exists a Prometheus reporter, maintained by the VoxPupuli community. However, it uses a dropzone directory of yaml files with a local node exporter, so it's not a very clean approach.
\nOn top of this, reports and their metrics are already exported to the PuppetDB, which provides its own API to access this data.
\nPrometheus PuppetDB Exporter is a simple go binary that can scrape the PuppetDB for report metrics for Prometheus. It runs independently of the Puppet stack, and can be tuned to collect various types of metrics:
\nThe exporter provides metrics in the form\npuppet_report_<type>\nfor each of these types.
# HELP puppetdb_exporter_build_info puppetdb exporter build informations\n# TYPE puppetdb_exporter_build_info gauge\npuppetdb_exporter_build_info{build_date="2019-02-18",commit_sha="XXXXXXXXXX",golang_version="go1.11.4",version="1.0.0"} 1\n# HELP puppetdb_node_report_status_count Total count of reports status by type\n# TYPE puppetdb_node_report_status_count gauge\npuppetdb_node_report_status_count{status="changed"} 1\npuppetdb_node_report_status_count{status="failed"} 1\npuppetdb_node_report_status_count{status="unchanged"} 1\n This makes it fully compatible with Vox Pupuli's reporter implementation.
\nThe exporter is provided as a Docker image, and is included by default in Camptocamp's PuppetDB Helm chart.
\nCoupled with (a slightly modified version of) Julien Pivotto's Puppet Report dashboard, you can make some pretty graphs from these metrics:
\n![\"Node](\"https://dev-to-uploads.s3.amazonaws.com/i/jczukvqf1fs7dau9jnfp.png\")
This post is also available on DEV.
\n "},{"url":"/posts/github-sponsors-and-dev-to-posts-51b1/","relativePath":"posts/github-sponsors-and-dev-to-posts-51b1.md","relativeDir":"posts","base":"github-sponsors-and-dev-to-posts-51b1.md","name":"github-sponsors-and-dev-to-posts-51b1","frontmatter":{"title":"💡 GitHub Sponsors and dev.to posts","stackbit_url_path":"posts/github-sponsors-and-dev-to-posts-51b1","date":"2020-07-02T05:52:51.122Z","excerpt":"GitHub Sponsors could be leveraged on dev.to to generate revenue ","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--4g3b1qoK--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://dev-to-uploads.s3.amazonaws.com/i/qdp4swhb2kngx4cmsowd.png","comments_count":3,"positive_reactions_count":5,"tags":["discuss","github","sponsors","meta"],"canonical_url":"https://dev.to/raphink/github-sponsors-and-dev-to-posts-51b1","template":"post"},"html":"Yesterday, I read a blog post by Caleb Porzio about making money from Open Source projects, in particular by leveraging the GitHub sponsors program.
\nHis conclusion is that the best way to use GitHub Sponsors is to make both free and paid educational content, such that free content attracts an audience, and then they want to support you to access the advanced, paid, content.
\nIt seems to me that the DEV community, being already connected to GitHub, would be a perfect place to implement this idea.
\nThere could be a tag in the meta parameters of a post, which would restrict its access (fully or partly, e.g. by hiding the end of the article like lots of newspapers do) to the author's GitHub sponsors:
\n# Restrict to sponsors of the raphink GitHub account\nrestrict_sponsors: raphink\n# Restrict to sponsors of tier $10 or above\nrestrict_sponsors: raphink/10\n What do you think? Is that a feature that would be interesting for the DEV community?
\nThis post is also available on DEV.
\n "},{"url":"/posts/integrating-prometheus-with-puppetdb-aom/","relativePath":"posts/integrating-prometheus-with-puppetdb-aom.md","relativeDir":"posts","base":"integrating-prometheus-with-puppetdb-aom.md","name":"integrating-prometheus-with-puppetdb-aom","frontmatter":{"title":"Integrating Prometheus with PuppetDB","stackbit_url_path":"posts/integrating-prometheus-with-puppetdb-aom","date":"2020-04-30T10:48:51.895Z","excerpt":"Many applications are not containerized, and we still need to monitor their nodes. Prometheus PuppetDB SD allows to discover nodes in the PuppetDB and generate Prometheus configurations automatically.","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--GyMJoqm7--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://www.camptocamp.com/wp-content/uploads/xbanner-1.png.pagespeed.ic.-LxmyH1pjm.webp","comments_count":0,"positive_reactions_count":0,"tags":["devops","showdev","puppet","opensource"],"canonical_url":"https://www.camptocamp.com/actualite/integrating-prometheus-with-puppetdb/","template":"post"},"html":"Most companies that have switched their deployments to containers have faced this issue: traditional monitoring systems just don't cut it when it comes to observability of containerized applications. Instead of focusing on nodes and applications running on them, the cluster approach to container orchestration systems requires to target application instances, which can run on multiple nodes —even several times on a single node— and typically have short life spans.
\nFortunately for us, Prometheus came early on in the ecosystem, providing an elegant solution to gather metrics from microservices and derive all sort of observability tools from them, including monitoring. Prometheus also allows to monitor the cluster nodes, by returning their metrics and aggregating them into views of their own. Problem solved, we can now get rid of our historical monitoring infrastructure. Or can we?
\n![](\"https://www.camptocamp.com/wp-content/uploads/prometheus-550x120.png\")
As much as we'd like to think all our applications are now containerized and all our machines are neutral cluster nodes in a large cattle, the reality is often very different. Most companies still have a large quantity of specialized machines---even snowflakes at times--- that are not taken into account by your latest Kubernetes cluster. Should we keep Nagios running for those, or is it possible to make them fit into the new paradigm?
\nFor those of us running Puppet, the PuppetDB has for years been a great source of information on nodes managed by Puppet. It contains facts, catalogs, reports and more for all the nodes in the fleet. Let's use this information to monitor the nodes and their services dynamically, using Prometheus!
\n![](\"https://www.camptocamp.com/wp-content/uploads/puppet-2-400x400.png\")
Prometheus PuppetDB SD allows to link PuppetDB with your Prometheus infrastructure. At a regular interval, it queries the PuppetDB and retrieves a list of targets. It then outputs a scrape configuration which Prometheus can use. Sounds simple? It really is!
\nThe Vox Pupuli Puppet community has a great module to manage Prometheus. The puppet-prometheus module works out of the box and allows to install and configure a Prometheus server. It also provides the\nprometheus::scrape_job\ndefined type to declare scrape jobs to be added to the server.
Declaring these scrape jobs as exported resources tags them as such in the PuppetDB, and they can then be realized on the Prometheus server. This is extremely useful, but only works when the Prometheus server is managed by Puppet, not when it is containerized! Prometheus PuppetDB SD fills this gap by scraping the exported resources from the PuppetDB and generating the Prometheus configurations, so you can get the best of both worlds: scrape jobs declared in Puppet alongside your applications, and a containerized Prometheus server!
\nThere's several ways to install Prometheus PuppetDB SD, but let's face it: if you're using Prometheus, you probably have a Kubernetes cluster already, so let's install it using Helm:
\n$ helm repo add camptocamp http://charts.camptocamp.com\n$ helm install camptocamp/prometheus-puppetdb-sd --version 2.0.3\n The following values should be provided:
\nprometheusPuppetdbSd.args.puppetdb.url\n: the PuppetDB URI
prometheusPuppetdbSd.args.prometheus.proxy-url\n: the Prometheus\nPush-proxy URL
prometheusPuppetdbSd.args.output.k8s-secret.secret-name\n: the name\nof the k8s secret used to output the Prometheus configuration
prometheusPuppetdbSd.args.output.k8s-secret.secret-key\n: the key in\nthe k8s secret used for the output file name
CACert\n: the CA certificate used to authenticate the PuppetDB
Cert\n: the certificate used to connect to the PuppetDB
Key\n: the private key used to connect to the PuppetDB
With the official Prometheus Operator, setting\nprometheusSpec.additionalScrapeConfigsExternal\nto\ntrue\nwill automatically configure Prometheus to mount the secret called\n{{ template \"prometheus-operator.fullname\" . }}-prometheus-scrape-confg\nand use the\nadditional-scrape-configs.yaml\nkey in it as additional configuration. This is thus the easiest way to configure Prometheus PuppetDB SD.
That's it, you're set!
\nDon't hesitate to provide feedback and pull requests on the GitHub repository!
\n\nThis post was originally published on https://www.camptocamp.com/actualite/integrating-prometheus-with-puppetdb/
\nThis post is also available on DEV.
\n "},{"url":"/posts/keep-an-eye-on-your-terraform-states-4lf5/","relativePath":"posts/keep-an-eye-on-your-terraform-states-4lf5.md","relativeDir":"posts","base":"keep-an-eye-on-your-terraform-states-4lf5.md","name":"keep-an-eye-on-your-terraform-states-4lf5","frontmatter":{"title":"Keep an eye on your Terraform states","stackbit_url_path":"posts/keep-an-eye-on-your-terraform-states-4lf5","date":"2020-05-08T08:11:27.873Z","excerpt":"About 4 years ago, we started using Terraform. Many things we were doing manually in the cloud at the time are now coded.","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--EFbU1JTl--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://www.camptocamp.com/wp-content/uploads/xterraform_bandeau.png.pagespeed.ic.neAGqH-_lX.webp","comments_count":0,"positive_reactions_count":6,"tags":["terraform","devops","showdev","opensource"],"canonical_url":"https://www.camptocamp.com/actualite/keep-an-eye-on-your-terraform-states/","template":"post"},"html":"This blog post was originally published on camptocamp.com
\nAbout 4 years ago, we started using Terraform. Many things we were doing manually in the cloud at the time are now coded. As a result, our Terraform base code now contains over a hundred states.
\nA lot of those resources already existed before, some managed by CloudFormation, others manually. Being able to import resources has helped a lot to integrate new Terraform code with existing infrastructure. We now have a unified system to control them, and most importantly to know who created them, how and why. Collaboration was made easier by using profiles instead of hardcoded credentials, the introduction of remote states stored on AWS S3, as well as state locks on DynamoDB.
\nWith all this, one thing remained: how do we keep an eye on all these states, resources and locks that are stored on AWS? Could there be a way to visualize and query them?
\n![\"Terraboard\"](\"https://raw.githubusercontent.com/camptocamp/terraboard/master/logo/terraboard_logo.png\"){kind=logo}
Terraboard was born in an attempt to bring an easy-to-use Web Interface for Terraform states.
\nIt currently supports states stored in AWS S3, as well as locks on DynamoDB. It features 4 views: overview, state view, compare view and search.
\nTerraboard requires an S3 bucket with versioning activated (for history and comparison between versions), as well as a PostgreSQL database, where all S3 states will be stored as a data cache.
\nTerraboard is comprised of two compontents:
\nThe overview is the landing page in Terraboard. It provides information about the most recent version of each state, along with the Terraform version used to apply it, its serial, the number of resources it features, and an activity sparkline. Clicking the sparkline lets you easily access any version of a state.
\nGraphs present statistics on the main resource types and Terraform versions used, as well as the number of number of states locked (if DynamoDB is configured).
\n![\"Main](\"https://www.camptocamp.com/wp-content/uploads/main-550x326.png\")
The State view presents details about a state file's resources. Resources are listed by module and can be filtered. A version selector lets you view historical data for the state.
\n![\"State](\"https://www.camptocamp.com/wp-content/uploads/state-550x328.png\")
While on the State view, you can pick a second version to compare with the current one. This computes differences between the two versions, which displays:
\n![\"Compare](\"https://www.camptocamp.com/wp-content/uploads/compare-550x330.png\")
If you've ever wondered in which Terraform state a node was managed, you can easily find this out in the Search view. The Search view lets you filter resources and their attributes by type, name, key or value, as well as Terraform version used.
\n![\"Search](\"https://www.camptocamp.com/wp-content/uploads/search-1-550x330.png\")
Are you ready to try Terraboard? If you're using Docker, this is very easy. All you need is a PostgreSQL database and your AWS credentials:
\ndocker run -d -p 8080:8080 \\\n -e AWS_REGION=<AWS_DEFAULT_REGION> \\\n -e AWS_ACCESS_KEY_ID=<AWS_ACCESS_KEY_ID> \\\n -e AWS_SECRET_ACCESS_KEY=<AWS_SECRET_ACCESS_KEY> \\\n -e AWS_BUCKET=<terraform-bucket> \\\n -e AWS_DYNAMODB_TABLE=<terraform-locks-table> \\\n -e DB_PASSWORD="mygreatpasswd" \\\n --link postgres:db \\\n camptocamp/terraboard:latest\n A Rancher template is also available in Camptocamp's Rancher Catalog, as well as a Helm Chart.
\nTerraboard is an open-source project and we heartily welcome all contributions to it. Don't hesitate to submit Pull Requests on GitHub.
\nYou are also welcome to join us on Gitter to discuss new ideas.
\nHappy Terraforming!
\nThis post is also available on DEV.
\n "},{"url":"/posts/march-cloud-native-romandie-meetup-o2f/","relativePath":"posts/march-cloud-native-romandie-meetup-o2f.md","relativeDir":"posts","base":"march-cloud-native-romandie-meetup-o2f.md","name":"march-cloud-native-romandie-meetup-o2f","frontmatter":{"title":"March Cloud Native Romandie Meetup","stackbit_url_path":"posts/march-cloud-native-romandie-meetup-o2f","date":"2021-04-01T13:44:49.095Z","excerpt":"The last Cloud Native Romandie Meetup took place on March 25th","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--md-eQ-xl--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cf6t3h6z8shvv26x7tqr.png","comments_count":0,"positive_reactions_count":0,"tags":["meetup","cloudnative","kubernetes","cicd"],"canonical_url":"https://dev.to/camptocamp-ops/march-cloud-native-romandie-meetup-o2f","template":"post"},"html":"Last week, we organized our last Cloud Native Romandie Meetup. Due to the current situation, this was an online event like the previous occurrences.
\nThe meetup was recorded and can be viewed again on YouTube.
\n\nFor this edition, we had presentations by CloudBees, Exoscale, and Camptocamp.
\nFrédéric Gibelin from CloudBees presented CloudBees CI, a solution that helps users scale their Jenkins Enterprise platform in the Cloud.
\n\nNext Pierre-Yves Ritschard and Mathieu Corbin from Exoscale presented SKS, a Kubernetes as-a-Service, operating the user’s cluster on their behalf and taking care of the underlying infrastructure.
\n\nTo finish, Raphaël Pinson presented Camptocamp's DevOps Stack project, a framework to deploy a standardized Kubernetes platform and its ecosystem, using a GitOps approach.
\n\nDon’t miss any more and join the Cloud Native Romandie meetup group. This way, you can be part of a local community and stay up-to-date on different Cloud Native technologies.
\nWe look forward to meeting and exchanging with you during our next virtual meetup scheduled for Thursday, June 17th.
\nAlso, if you are keen to present a technology or you would like to see more of a technology, please let us know, we would be happy to support your interest. If it interests you, it also interests the community.
\nThis post is also available on DEV.
\n "},{"url":"/posts/open-source-standards-and-technical-debt-2g1/","relativePath":"posts/open-source-standards-and-technical-debt-2g1.md","relativeDir":"posts","base":"open-source-standards-and-technical-debt-2g1.md","name":"open-source-standards-and-technical-debt-2g1","frontmatter":{"title":"Open Source, Standards, and Technical Debt","stackbit_url_path":"posts/open-source-standards-and-technical-debt-2g1","date":"2021-02-03T11:10:41.220Z","excerpt":"As software needs evolve, technological evolution implies Technical Debt. Open Source can help mitigate Technical Debt by influencing on standards.","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--OoTHD5Xo--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://dev-to-uploads.s3.amazonaws.com/i/a1ozenortfr4pe0d1vad.png","comments_count":3,"positive_reactions_count":4,"tags":["devops","agile","opensource","productivity"],"canonical_url":"https://www.camptocamp.com/en/news-events/open-source-standards-and-technical-debt","template":"post"},"html":"Twenty years ago, Camptocamp was a pioneer company in Open Source adoption. Nowadays, Open Source has become mainstream and the vast majority of the industry agrees on the many benefits of its practices. In fact, the Open Source model has become a de facto standard in some fields such as Web Frontend development.
\nMany companies make an increasing use of Open Source software in their infrastructure and development stacks, and there are countless proven reasons for doing so, such as standard formats or security by openness, to name just a couple.
\nIn spite of these benefits, companies openly contributing —let alone Open Sourcing their own projects— are still somehow not very common, and most firms think of Open Source purely as a consumer’s benefit.
\n![\"Open](\"https://dev-to-uploads.s3.amazonaws.com/i/dqdagbzcqm5kgkbcx4qz.png\")
For years, I used to think the best argument in favor of contributing to existing projects was maintenance and compatibility. If I fork a project and add functionality to it, there is a risk that my changes will become increasingly hard to maintain as time goes by. If the core developers of the program are aware of my changes and actively intend to go along with them, this risk will be greatly reduced.
\nSo contributing my changes ensures they will stay compatible with the base code as time goes by. There might even be improvements to my code if more people encounter a similar need in the future, and decide to build on top of my changes.
\nToday, however, I believe the example I have just given is a specific case of a more general rule, which encompasses more pragmatic reasons to contribute code as Open Source. This more general context is linked to the concept of Technical Debt: the idea that technical decisions imply a hidden cost (a “debt”) that will have to be paid in the future in order to catch up with state-of-the-art technology.
\nMinimizing technical debt is a vast —and at times conflicting— subject. However, I think it is safe to assume that one way to reduce its risk is to stick to standards. The closer a project sticks to industry standards, the less likely it will have to be ported to another technological stack in the foreseeable future.
\nWhen faced with a missing feature, most people’s reflex might be to start building a specific component to meet their use case. In the words of Strategy Theorist Simon Wardley, they’ll be shifting this component to the Genesis stage, making it more unpredictable —or even erratic—, less standard, and thus more prone to building up technical debt in time.
\nThere is another way though. If my need is not met, and it is in fact a valid need (which is a very important question to ask in the first place), then other people might have this need in the future. When they do, someone, somewhere, will create a new standard for this need. When this new standard becomes enforced, then will my specific component’s debt become obvious.
\nSo what if, instead of building a specific component to make up for the lack of standard, I set the new standard myself? Open Source lets you do just that! It gives you the opportunity to be the first one providing an open implementation to a generic need, and the chance to make it into the new standard. If that new standard catches on, you have not only solved your problem, but you also haven’t accumulated technical debt. In fact, you’re ahead of the other users, because you set the new standard.
\nObviously, the majority of organizations can't afford to have engineers focusing on IETF RFCs or moving ISO standards to fit our needs.
\nHowever, a standard doesn't have to be that complicated. Let’s say I use this popular CLI tool, but I need to specify an option which doesn’t exist yet. I could hack something around the generation of its configuration file to produce the options I need. Or I could patch that tool and add a new flag for my needs, and contribute that change back to the project. Chances are, if I need this option, some else does too.
\nNow, every time someone has the need for that option, they’ll be using my new flag. I’ve contributed a new standard, and I haven’t made any technical debt on my side.
\nIt’s not the size of the steps that matters, it’s really the direction in which you take them.
\n![\"Start](\"https://dev-to-uploads.s3.amazonaws.com/i/8z3stx204q4gut3w4coq.png\")
Open Source is not just a philosophy. It encompasses licensing issues, technology standards, culture, and much more.
\nAt Camptocamp, we’ve been committed to the Open Source approach for years.
\nThis means we have a habit of solving problems in generic terms and building new standards.
\nIt also means we have contacts in many Open Source communities, which allow us to brainstorm ideas and quickly contribute to projects, ensuring a fast feedback loop on our work.
\nWhen we implement Open Source software for our clients, we actively seek to limit technological debt. Because we believe in a world of standards, we don’t want our clients to feel entirely stuck with a technological stack in the future. Or even with us!
\nThis post is also available on DEV.
\n "},{"url":"/posts/recognizing-faces-in-historical-photographs-3ikc/","relativePath":"posts/recognizing-faces-in-historical-photographs-3ikc.md","relativeDir":"posts","base":"recognizing-faces-in-historical-photographs-3ikc.md","name":"recognizing-faces-in-historical-photographs-3ikc","frontmatter":{"title":"Recognizing faces in historical photographs","stackbit_url_path":"posts/recognizing-faces-in-historical-photographs-3ikc","date":"2020-05-03T20:10:58.839Z","excerpt":"Using machine learning to identify people in historical photographs","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--xqA9hriG--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://dev-to-uploads.s3.amazonaws.com/i/zkzysi903rs0m6w8rvy1.png","comments_count":0,"positive_reactions_count":6,"tags":["machinelearning","facerecognition","showdev","opensource"],"canonical_url":"https://dev.to/raphink/recognizing-faces-in-historical-photographs-3ikc","template":"post"},"html":"Genealogy has been one of my main personal activities for years. As part of my research, I've collected old pictures of family members that cousins I met were kind enough to send me (usually in scanned form, although at times I was actually given the custody of original photographs).
\nIn the last few years, I've added these photographs to Google Photos to take advantage of the face recognition features. It's allowed me to quickly find pictures of people, and it has helped me to identify people in pictures with the help of AI.
\n![\"Google](\"https://dev-to-uploads.s3.amazonaws.com/i/wtowmhpvyc51m79yef5j.png\")
\nGoogle Photos helps me keep track of known people in photographs
However useful this has been though, I've felt for some time that the scope was too narrow. I've found pictures of my great-grandfather and his associate in books and newspapers, and there's probably more I haven't seen yet.
\nFurthermore, there's people in my collection that I haven't identified yet. Somewhere, somehow, I'm sure there's descendants of these people who have portraits of them, and would love to get more pictures of their ancestors. I have occasionally been able to identify them with notes in the back of pictures, but there's still a lot left to put a name on.
\nSo I've been thinking… What if I could have a system similar to Google Photos face grouping and identification, but at a much more global scale?
\nThe time seems ripe for this:
\nA few months ago, I've started playing with AWS Rekognition to see what I could get out of my own personal collection. Encouraged by the results, I launched a little PoC project, which can be found at:
\nhttps://raphink.github.io/find-my-ancestor/
\n![\"President](\"https://dev-to-uploads.s3.amazonaws.com/i/79uz5005ebjyex7knfwz.png\")
\nPresident Paul Kruger found by the AI among his family
In this project, I picked public Flickr collections featuring historical photographs from all over the world. I scanned a few million pictures and stored face metadata about them in AWS Rekognition. I then built a simple web UI to query this database from a given picture.
\nThe code (both Ruby scripts and web interface) can be found on GitHub:
\n\nI've communicated about this project on various Genealogy websites and groups. Unfortunately, the results were not so great. Apart from celebrities and royalties, it's hard to identify random people in a database of \"only\" a million faces, though I am quite sure the algorithm did identify my great-great-uncle in two pictures from the Boer War.
\nMyHeritage recently worked with AI developer Jason Antic to provide an amazing colorization algorithm.
\nMost of these genealogical websites provide some kind of hinting system, which send you regular notifications of:
\nMy goal would thus be to provide a new kind of hint, in the form of photographs matching known portraits of people in your tree.
\nAfter giving it some thinking, I'm afraid though that the database I have built in AWK Rekognition won't be of much help. It seems I should be using another kind of algorithm to group faces by known person in order to improve matching.
\nI'd love to this see project get somewhere. There's so many people who could be identified… soldiers in WWI/WWII pictures, lost family members in concentration camps, and many other unsolved mysteries…
\nDo you AI experts have any tips to help me continue this project?
\nThis post is also available on DEV.
\n "},{"url":"/posts/representing-technical-skills-on-a-timeline-1mk1/","relativePath":"posts/representing-technical-skills-on-a-timeline-1mk1.md","relativeDir":"posts","base":"representing-technical-skills-on-a-timeline-1mk1.md","name":"representing-technical-skills-on-a-timeline-1mk1","frontmatter":{"title":"Representing technical skills on a timeline","stackbit_url_path":"posts/representing-technical-skills-on-a-timeline-1mk1","date":"2020-05-11T15:19:14.715Z","excerpt":"Several ways to display technical skills on a timeline","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--5QxM9CRu--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://dev-to-uploads.s3.amazonaws.com/i/0kwh14busu2ckvvmofwp.png","comments_count":1,"positive_reactions_count":7,"tags":["jquery","latex","showdev","opensource"],"canonical_url":"https://dev.to/raphink/representing-technical-skills-on-a-timeline-1mk1","template":"post"},"html":"CVs and other websites presenting technical skills often lack a time dimension, allowing to know when and for how long a technology has been used.
\nAbout 8 years ago, I wanted to add a visual representation of my experience on my PDF CV.
\nSince I already used LaTeX with the excellent moderncv class, I wanted the solution to extend on that class. TeX StackExchange did not disappoint (they never do) and this gave birth to the \nmoderntimeline\nLaTeX package which I have been maintaining since.
![\"Moderntimeline](\"https://dev-to-uploads.s3.amazonaws.com/i/4ae3bwwiymnffrljjypc.png\")
To this day I still use this solution on my CV.
\nSince then, a template has even been added to Overleaf to make it easier!
\n![\"Template](\"https://dev-to-uploads.s3.amazonaws.com/i/hvcmi91fhd8eaqex8u8c.png\")
The CV timeline is still not enough to present the data I wish to display, which is the temporal evolution of technical skills.
\nAmong the many websites which analyze public code repositories to get metrics out of them, OpenHub (previously Ohloh) is very interesting because it presents a timeline of languages used in projects.
\nHere's an example with my profile, where you can identify clear periods: a lot of LaTeX (dark blue) in the first years (when I edited books), then Augeas (light grey), mostly Ruby (red) between 2012 and 2015, then mainly Go (purple).
\n![\"OpenHub](\"https://dev-to-uploads.s3.amazonaws.com/i/0kwh14busu2ckvvmofwp.png\")
Not every tech skill can be measured with a number of code lines though.\nSo in 2013, I switched my main CV page to a temporal skills view.
\n![\"Skills](\"https://dev-to-uploads.s3.amazonaws.com/i/8ld9549ukxidnb7i7jjv.png\")
This uses vis.js to build a table of skills from a JSON file, e.g.:
\n[\n {"id": "Orange", "content": "<img src='img/orange.png' class='logo' /><b>Orange Portails</b><br />Systems Engineer", "start": "2006-06-01", "end": "2012-03-01", "type": "background", "className": "orange"},\n {"id": "Camptocamp", "content": "<img src='img/camptocamp.png' class='logo' /><b>Camptocamp</b><br />Infrastructure Developer", "start": "2012-03-01", "type": "background", "className": "camptocamp"},\n\n {"group": "provisioning", "content": "Debian FAI", "start": "2006-06-01", "end": "2012-03-01", "className": "contributed"},\n {"group": "provisioning", "content": "Kickstart", "start": "2006-06-01", "className": "implemented"},\n {"group": "provisioning", "content": "Terraform", "name": "terraform", "start": "2016-05-01", "className": "contributed"}\n]\n This JSON file is parsed and displayed on the page. Each skill can be assigned an icon as well as additional information. The skill bar can be clicked to display this information, taken from the\nskills/\ndirectory and documented in Markdown.
![\"Details](\"https://dev-to-uploads.s3.amazonaws.com/i/xe2j9wktc28leawnu9ee.png\")
The code is open-source and can be forked on GitHub. Just check the\ngh-pages\nbranch:
As usual, pull requests are welcome if you find nice ways to improve this!
\nThis post is also available on DEV.
\n "},{"url":"/posts/taming-puppetserver-6-pt-ii-garbage-collection-2oh2/","relativePath":"posts/taming-puppetserver-6-pt-ii-garbage-collection-2oh2.md","relativeDir":"posts","base":"taming-puppetserver-6-pt-ii-garbage-collection-2oh2.md","name":"taming-puppetserver-6-pt-ii-garbage-collection-2oh2","frontmatter":{"title":"Taming Puppetserver 6 Pt II: Garbage Collection","stackbit_url_path":"posts/taming-puppetserver-6-pt-ii-garbage-collection-2oh2","date":"2020-05-15T10:49:37.667Z","excerpt":"PuppetServer can be spending a lot of time doing gargage collection, which impacts its performance","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--0QPlyKrh--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://dev-to-uploads.s3.amazonaws.com/i/1wtluh7qosm01n29xmgq.png","comments_count":0,"positive_reactions_count":3,"tags":["puppet","observability","java","garbagecollection"],"canonical_url":"https://dev.to/camptocamp-ops/taming-puppetserver-6-pt-ii-garbage-collection-2oh2","template":"post"},"html":"Now that our internal Puppet Infrastructure is migrated to Puppet 6 and tuned, it was time to switch a second infra to it.
\nYesterday, I migrated our second infrastructure, and started seeing more issues. The rules-of-thumb from last post were useful, but I still needed to upgrade available memory to make up for a lack of computing power (probably to be imputed to the underlying IaaS throttling virtual CPUs).
\nAnd then, a Puppetserver crashed with a\nGC overhead limit exceeded\nerror. This error happens when the CPU spends more than 98% performing garbage collection.
Looking at our Grafana dashboard, I realized we had no metrics about garbage collection, so I added a graph with two metrics:
\njvm_gc_collection_seconds_sum\nis a cumulative counter)jvm_gc_collection_seconds_count\nis also a cumulative counter)The formulas are as follow:
\ntime per request {{gc}}\n:\nrate(jvm_gc_collection_seconds_sum{job=\"puppetserver\"}[1m])/rate(jvm_gc_collection_seconds_count{job=\"puppetserver\"}[1m])
rate {{gc}}\n:\nrate(jvm_gc_collection_seconds_sum{job=\"puppetserver\"}[1m])
![\"Graph](\"https://dev-to-uploads.s3.amazonaws.com/i/euwyfd4972ggylucm4n3.png\")
I then looked at the graphs around the time when the\nGC overhead limit exceeded\nerror happened:
![\"GC](\"https://dev-to-uploads.s3.amazonaws.com/i/w6k51xxcgoucexyieap3.png\")
Yes, I had a problem indeed. I restarted the Puppetservers and this hasn't happened since. However, the rates for PS MarkSweep have kept pretty high still. Here's the last 15 minutes as I'm writing:
\n![\"Standard](\"https://dev-to-uploads.s3.amazonaws.com/i/y93wzup9e7wglf4ectng.png\")
In comparison, the infrastructure I upgraded last week is faring much better, with GC rates well under 10%:
\n![\"Standard](\"https://dev-to-uploads.s3.amazonaws.com/i/466p0q75k2irwheuze22.png\")
In addition to a high rate spent performing garbage collection on the PS MarkSweep GC threads, I also noticed that the mean GC time for PS MarkSweep is pretty high, too, at around 2 to 3s. The values are slightly lower (a bit under 2s) on my first infrastructure.
\nAll in all, it seems PS MarkSweep garbage collection is to blame. It tends to take lots of CPU, for long periods of time.
\nThe good news is: PS MarkSweep is a legacy garbage collector, and it's not too hard to get rid of it, since OpenJDK 11 replaces it with the G1 Young Generation garbage collector by default.
\nThe official puppetserver Docker image installs the\npuppetserver\npackage, which pulls\nopenjdk-8-jre-headless\nas a dependency. OpenJDK 11 is also officially supported starting with Puppet 6.6, but the package doesn't allow to install it instead of OpenJDK 8. So for now, I'll just derive an image and install\nopenjdk-11-jre-headless\nin addition to OpenJDK8 and let Ubuntu update the alternative for Java automatically.
The following graph shows the difference in GC time between PS MarkSweep and G1, following the upgrade to OpenJDK 11:
\n![\"PS](\"https://dev-to-uploads.s3.amazonaws.com/i/1wtluh7qosm01n29xmgq.png\")
And here's what GC looks like after a good warm up:
\n![\"New](\"https://dev-to-uploads.s3.amazonaws.com/i/hvr49m18eeidpxt31u7k.png\")
From 2s to 80ms, that's a great improvement if you ask me!
\nThis post is also available on DEV.
\n "},{"url":"/posts/tracing-x-to-my-4th-great-grandmother-2af9/","relativePath":"posts/tracing-x-to-my-4th-great-grandmother-2af9.md","relativeDir":"posts","base":"tracing-x-to-my-4th-great-grandmother-2af9.md","name":"tracing-x-to-my-4th-great-grandmother-2af9","frontmatter":{"title":"Tracing X to my 4th great-grandmother","stackbit_url_path":"posts/tracing-x-to-my-4th-great-grandmother-2af9","date":"2020-06-03T22:23:33.398Z","excerpt":"X chromosomes have a specific inheritance pattern which often allow to narrow family branches when looking for relationships hypothesis","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--n-mB5MIa--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://upload.wikimedia.org/wikipedia/commons/thumb/5/59/Ideogram_house_mouse_chromosome_X.svg/1280px-Ideogram_house_mouse_chromosome_X.svg.png","comments_count":0,"positive_reactions_count":2,"tags":["dna","genealogy","dnapainting"],"canonical_url":"https://dev.to/raphink/tracing-x-to-my-4th-great-grandmother-2af9","template":"post"},"html":"DNA tests are fun. They can give you a hint on your origins (though the results depend a lot on the data sets from the company providing them), get you in touch with cousins (or even closer relatives) you didn't know about, confirm genealogy hypothesis, and much more...
\nOne thing that is interesting to do with DNA results is to trace known DNA segments to a known ancestor. This is not always possible, and usually requires several triangulated matches on that segment to ensure where it comes from.
\nIn my DNA results, I have a rather large match on my X chromosome. This match is about 40cM long (for a total length of about 196cM at FamilyTreeDNA).
\n![\"A](\"https://dev-to-uploads.s3.amazonaws.com/i/690rgl67j0nnunn5shm0.png\")
Where could this segment have come from? Is there a way to tell?
\nIn autosomal tests, the X chromosome is often included, but it plays by\nits own rules. This is because X is transmitted in a funny way, due to\nthe fact that males only have one X chromosome (and one Y chromosome),\nwhile females have two (and no Y chromosome).
\nHere's an illustration from Wikipedia explaining how this affects the inheritance rules of that special chromosome:
\n![\"X](\"https://upload.wikimedia.org/wikipedia/commons/thumb/e/ed/X_chromosome_ancestral_line_Fibonacci_sequence.svg/1920px-X_chromosome_ancestral_line_Fibonacci_sequence.svg.png\")
As you can see, the consequence is pretty simple:
\nAs a result, a segment of an X chromosome can travel through both men and women (unlike segments on the Y chromosome, which travel only through men), but can never be carried by two men in a row.
\nIn other words, since I am a man, I cannot have gotten this segment from:
\nNow I'm quite lucky, as I happen to know the person I match on 40cM on that X chromosome, even though the match is quite distant:
\nSo, unless a closer path can be found in the future (which is unlikely), the inheritance hypothesis checks with both sides of the tree, confirming the chain of ancestors:
\n![\"Tracing](\"https://dev-to-uploads.s3.amazonaws.com/i/yd2eyu0ihscnz7emcd51.png\")
You can see how that X segment was transmitted from my 4th great-grandmother Luna Mondolfo (born around 1790) all the way to myself (on the left) and my DNA match (on the right).
\nOnce this is confirmed, it also allows to consider that any other match triangulating on this X segment is more likely linked to this side of the family, known and verified until the 18th century.
\nA great way of tracing known DNA segments is to use DNA Painter, a free website where you can gather segment information from various companies (FTDNA, Ancestry, GedMatch, MyHeritage, etc.) to \"paint\" your chromosomes.
\nHere's an example showing my maternal labels, with chromosome X at the bottom:
\n![\"DNA](\"https://dev-to-uploads.s3.amazonaws.com/i/vejbspsiy6tmsg8wcpu1.png\")
Did you make interesting finds in DNA tests as well?
\nThis post is also available on DEV.
\n "},{"url":"/posts/unshallowing-a-git-repository-24nd/","relativePath":"posts/unshallowing-a-git-repository-24nd.md","relativeDir":"posts","base":"unshallowing-a-git-repository-24nd.md","name":"unshallowing-a-git-repository-24nd","frontmatter":{"title":"Unshallowing a Git repository","stackbit_url_path":"posts/unshallowing-a-git-repository-24nd","date":"2020-05-08T07:36:38.220Z","excerpt":"GitLab allows to perform shallow repository clones (and it seems to be the default in recent versions...","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--nIHJww2i--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://dev-to-uploads.s3.amazonaws.com/i/nsbbm80zgqqypxyqtx1d.png","comments_count":0,"positive_reactions_count":1,"tags":["devops","git","cicd","todayilearned"],"canonical_url":"https://dev.to/camptocamp-ops/unshallowing-a-git-repository-24nd","template":"post"},"html":"GitLab allows to perform shallow repository clones (and it seems to be the default in recent versions from what I can tell).
\nIn order to run r10k, I need a full repository though, because r10k will copy it to cache and use this copy as a reference. This is what happens when you use a shallow repository:
\n [2020-05-08 06:53:15 - DEBUG] Replacing /etc/puppetlabs/code/environments/modulesync_update and checking out modulesync_update\n [2020-05-08 06:53:56 - ERROR] Command exited with non-zero exit code:\n Command: git clone /builds/camptocamp/is/puppet/puppetmaster-c2c /etc/puppetlabs/code/environments/modulesync_update --reference /etc/puppetlabs/code/cache/-builds-camptocamp-is-puppet-puppetmaster-c2c\n Stderr:\n Cloning into '/etc/puppetlabs/code/environments/modulesync_update'...\n fatal: reference repository '/etc/puppetlabs/code/cache/-builds-camptocamp-is-puppet-puppetmaster-c2c' is shallow\n Exit code: 128\n [2020-05-08 06:53:56 - DEBUG] Purging unmanaged environments for deployment...\n Git provides a\nfetch --unshallow\ncommand which solves the problem, so we just need to run\ngit fetch --unshallow\nin the repository before running r10k.
However, some of our (older) GitLab installs don't make shallow clones. Instead, they make full clones with a single detached branch, so we need\nfetch --all\ninstead.
In order to have it work in all configurations, I'm ending up running:
\ngit fetch --unshallow || git fetch --all\n And then run r10k on the repository.
\nThis post is also available on DEV.
\n "},{"url":"/posts/automated-puppet-impact-analysis-1c1/","relativePath":"posts/automated-puppet-impact-analysis-1c1.md","relativeDir":"posts","base":"automated-puppet-impact-analysis-1c1.md","name":"automated-puppet-impact-analysis-1c1","frontmatter":{"title":"Automated Puppet Impact Analysis","stackbit_url_path":"posts/automated-puppet-impact-analysis-1c1","date":"2020-05-07T20:49:52.016Z","excerpt":"Using GitLab Pipelines and Catalog Diff to preview changes between two branches in a merge request","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--X2kztkXc--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://www.camptocamp.com/wp-content/uploads/xformations_puppet1-720x400.png.pagespeed.ic.UU2oY1Zlj8.webp","comments_count":4,"positive_reactions_count":9,"tags":["puppet","devops","codequality","showdev"],"canonical_url":"https://dev.to/camptocamp-ops/automated-puppet-impact-analysis-1c1","template":"post"},"html":"In last week's post, I presented how to set up Puppet Catalog Diff to diff between two Puppet environments.
\nWouldn't it be great if this tool could be used to perform automatic impact analysis before merging a Git branch (aka Merge Request or Pull Request)? Well, it can.
\nOur current set up is based on RedHat OpenShift and GitLab.\nThis is however easily portable to other installation choices.
\nThe Puppet infrastructure is currently running in OpenShift, using our series of Puppet Helm Charts for Puppetserver, PuppetDB, Puppetboard and Puppet Catalog Diff Viewer.
\n![\"Puppet-related](\"https://dev-to-uploads.s3.amazonaws.com/i/dn3718sndgu4zyrtgklv.png\")
We are in the process of migrating from Puppet 5 to Puppet 6, so we currently have two Puppetserver charts deployed, one for each version. The\npuppetserver\nservice points to two Puppet 5 pods, while the\npuppetserver6\nservice points to two Puppet 6 pods.
We have passthrough OpenShift routes sitting in front of the services to expose them to the rest of the infra (on port 443 instead of 8140).
\nPuppet code deployment is done using a GitLab Runner chart whose deployment mounts the Puppetcode volume (PVC from the Puppetserver deployment). We then run r10k in a GitLab pipeline every time a branch is pushed.
\nWe also lint the code before deploying it, using the Onceover Code Quality plugin.
\n![\"Deployment](\"https://dev-to-uploads.s3.amazonaws.com/i/t6o39hw6zpxdq27uhhmc.png\")
Here's what it looks like in\n.gitlab-ci.yml\n:
---\nstages:\n - lint\n - deploy\n\n.create_r10k_yaml: &create_r10k_yaml |\n cat << EOF > /tmp/r10k.yaml\n ---\n :cachedir: /etc/puppetlabs/code/cache\n\n :sources:\n :main:\n remote: $CI_PROJECT_DIR\n basedir: /etc/puppetlabs/code/environments\n EOF\n\nlinting-puppet-hiera:\n image: camptocamp/onceover-codequality:latest\n stage: lint\n script:\n - 'onceover run codequality --no_docs'\n tags:\n - puppetmaster\n rules:\n # Skip linting if the commit message contains "[skip lint]"\n - if: '$CI_COMMIT_MESSAGE !~ /\\[skip lint\\]/'\n\nr10k-deploy:\n image: puppet/r10k:3.1.0\n stage: deploy\n tags:\n # Select GitLab runner from the Puppet OpenShift env (which mounts Puppetcode)\n - puppetmaster\n before_script:\n - while [ -f /etc/puppetlabs/code/r10k.lock ]; do echo -n "Waiting for lock from "; cat /etc/puppetlabs/code/r10k.lock || echo; sleep 2; done\n - hostname -f > /etc/puppetlabs/code/r10k.lock\n script:\n - umask 0002\n # Git https secrets are mounted in the GitLab runner\n - ln -s /secrets/.netrc ~/\n - *create_r10k_yaml\n - git fetch --unshallow\n - 'git branch -r | grep -v "\\->" | while read remote; do git branch --track "${remote# origin/}" "$remote"; done'\n - r10k deploy --color -c /tmp/r10k.yaml environment ${CI_COMMIT_REF_NAME} -p --verbose=debug\n - puppet generate types --environment ${CI_COMMIT_REF_NAME}\n after_script:\n - rm -f /etc/puppetlabs/code/r10k.lock\n When a Merge Request is open, we want to analyse the impact it will have before we can merge it. This is where Catalog Diff plays a big role.
\nUnless you have a huge Puppet infrastructure, Catalog Diff is quite heavy to launch, as it will request lots of catalogs in a small amount of time.
\nThe new\n--old_catalog_from_puppetdb\noption introduced in version 1.7.0 reduces the load by half by getting the \"from\" catalogs from PuppetDB, but it's still kind of a large batch of requests to the Puppet servers.
For this reason, we run Catalog Diff only on demand, as a manual task. Lint and Deploy are run a second time, to make them mandatory passing steps before a merge can be validated.
\n![\"MR](\"https://dev-to-uploads.s3.amazonaws.com/i/xlzb38uvg70hndubvrze.png\")
Here's the setup:
\n.create_puppetdb_conf: &create_puppetdb_conf |\n cat << EOF > /etc/puppetlabs/puppet/puppetdb.conf\n [main]\n server_urls = https://puppetdb:8081\n EOF\n\n.create_csr_attributes_yaml: &create_csr_attributes_yaml |\n cat << EOF > /etc/puppetlabs/puppet/csr_attributes.yaml\n ---\n custom_attributes:\n # Our autosign script uses hashed secrets based on a psk,\n # the certname and the environment coded in the certificate\n 1.2.840.113549.1.9.7: '$(echo -n "$psk/$(puppet config print certname)/production" | openssl dgst -binary -sha256 | openssl base64)'\n extension_requests:\n # We use the pp_authorization=catalog extension to set up auth.conf for v4/catalog\n 1.3.6.1.4.1.34380.1.3.1: 'catalog'\n 1.3.6.1.4.1.34380.1.1.12: 'production'\n EOF\n\n.cleanup_cert: &cleanup_cert |\n curl -s -X DELETE \\\n "Accept:application/json" -H "Content-Type: text/pson" \\\n --cacert "/etc/puppetlabs/puppet/ssl/certs/ca.pem" \\\n --cert "/etc/puppetlabs/puppet/ssl/certs/$(puppet config print certname).pem" \\\n --key "/etc/puppetlabs/puppet/ssl/private_keys/$(puppet config print certname).pem" \\\n "https://puppetserver:8140/puppet-ca/v1/certificate_status/$(puppet config print certname)?environment=production"\n\n\ncatalog-diff:\n image: puppet/puppet-agent:6.15.0\n stage: diff\n tags:\n # Select GitLab runner in Puppet OpenShift env to get direct access to services\n - puppetmaster\n script:\n - apt update\n - apt install -y locales puppetdb-termini\n - locale-gen en_US.UTF-8\n - *create_puppetdb_conf\n - *create_csr_attributes_yaml\n # Generate a certificate and get it signed\n - puppet ssl submit_request --ca_server puppetserver --certificate_revocation=false\n # We currently diff with puppetserver6 for the migration\n - puppet catalog --environment ${CI_MERGE_REQUEST_SOURCE_BRANCH_NAME} --certificate_revocation=false diff puppetserver:8140/${CI_MERGE_REQUEST_TARGET_BRANCH_NAME} puppetserver6:8140/${CI_MERGE_REQUEST_SOURCE_BRANCH_NAME} --show_resource_diff --changed_depth 1000 --content_diff --old_catalog_from_puppetdb --certless --threads 4 --output_report /catalog-diff/mr_${CI_MERGE_REQUEST_IID}_${CI_JOB_ID}.json\n after_script:\n # We have configured our auth.conf to allow nodes to clean their own cert, see https://dev.to/camptocamp-ops/automatic-renewal-of-puppet-certificates-28pm\n - *cleanup_cert\n - echo "You can view the report details at https://puppetdiff.example.com/?report=mr_${CI_MERGE_REQUEST_IID}_${CI_JOB_ID}"\n # Post a comment on the Merge Request\n - 'curl -k -X POST -H "Private-Token: $CI_BOT_TOKEN" -d "body=You can view the Catalog Diff report details at https://puppetdiff.example.com/?report=mr_${CI_MERGE_REQUEST_IID}_${CI_JOB_ID}" $CI_API_V4_URL/projects/$CI_PROJECT_ID/merge_requests/$CI_MERGE_REQUEST_IID/notes'\n # Allow failure so the Merge Request can be validated even without catalog diff\n allow_failure: true\n rules:\n - if: '$CI_MERGE_REQUEST_ID'\n when: manual\n variables:\n LANG: en_US.UTF-8\n LC_ALL: en_US.UTF-8\n A few notes on that setup:
\nPuppetDB is accessed via SSL. Since we have valid certificates to access the Puppet server, we might as well, but 8080 is ok as well if you have that possibility.
\nWe use an autosign script to sign certificates using a PSK (which we hash). If it's easier for you, you could also inject a valid key and certificate into the build instead of a PSK.
\nIf you don't generate a certificate, you don't need the cleanup step either.
\nThe reports are saved to the\n/catalog-diff\ndirectory, which is mounted in the runner from the Puppet Catalog Diff Viewer PVC. This way, reports are accessible directly in the viewer by passing their name in the query string.
The Merge Request curl request requires passing a\nCI_BOT_TOKEN\nvariable to the build. We currently set one in the build variables, using a robot GitLab account. If you have a GitLab Silver or greater plan, you can use the\nCI_JOB_TOKEN\nvariable instead.
Here are some screenshots of a typical workflow.
\n![\"Validated](\"https://dev-to-uploads.s3.amazonaws.com/i/b7djk6oti87cf1iatsap.png\")
The Merge Request validated, with the comment left by the bot after the Catalog Diff build was run (see the 3 steps on line 3)
\n![\"Puppet](\"https://dev-to-uploads.s3.amazonaws.com/i/a87ep2w72mdbvkrdfsv8.png\")
Viewing the report generated by the Puppet Catalog Diff run
\nHere's a video demo of the setup described above:
\n\nThis set up allows us to:
\nAs stated in the previous blog post, this doesn't account for every change, since changes in plugins (facts, types & providers, Augeas lenses, etc.) can also impact servers but won't be seen in catalog diffs.
\nThis post is also available on DEV.
\n "},{"url":"/posts/cleaning-up-puppet-code-4da2/","relativePath":"posts/cleaning-up-puppet-code-4da2.md","relativeDir":"posts","base":"cleaning-up-puppet-code-4da2.md","name":"cleaning-up-puppet-code-4da2","frontmatter":{"title":"Cleaning up Puppet Code","stackbit_url_path":"posts/cleaning-up-puppet-code-4da2","date":"2020-04-29T10:54:01.869Z","excerpt":"Code quality is important to ensure style consistency and easy maintenance. Puppet-lint, Onceover and puppet-ghostbuster help ensure Puppet code quality.","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--X2kztkXc--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://www.camptocamp.com/wp-content/uploads/xformations_puppet1-720x400.png.pagespeed.ic.UU2oY1Zlj8.webp","comments_count":1,"positive_reactions_count":7,"tags":["puppet","devops","codequality","opensource"],"canonical_url":"https://www.camptocamp.com/actualite/cleaning-up-puppet-code/","template":"post"},"html":"After months and years of using Puppet, the code base becomes increasingly complex and cluttered. How can you ensure its quality, as well as clean up unused code?
\nIn the Puppet world, puppet-lint is the reference for code quality. It is used as a standard to check that modules follow the style guide, ensuring consistency in coding style and practices. puppet-lint can also be used in your control repository, to check your private modules (such as roles & profiles).
\nThere's at least three ways of achieving this: using PDK, a\nRakefile\n, or onceover along with its code quality plugin.
PDK, the standard tool to manage Puppet modules in a standard way, also works with control repositories. Once installed, you can convert your control repository and run validation tests with:
\n$ pdk convert\n$ pdk validate\n The\nRakefile\nmethod is a easy way to automate\npuppet-lint\n:
Rake::Task[:lint].clear\nPuppetLint::RakeTask.new :lint do |config|\n config.ignore_paths = [\n 'modules/**/*.pp',\n 'vendor/**/*',\n ]\n config.disable_checks = [\n '80chars',\n 'documentation',\n ]\n config.fail_on_warnings = true\n config.fix = true if ENV['PUPPETLINT_FIX'] == 'yes'\nend\n Add a\nGemfile\nin your repository to install\npuppet-lint\n:
source ENV['GEM_SOURCE'] || "https://rubygems.org"\n\ngroup :development, :test do\n gem 'rake', :require => false\n gem 'puppet-lint', :require => false\n \n # Other lint plugins (optional)\n gem 'puppet-lint-spaceship_operator_without_tag-check', :require => false\n gem 'puppet-lint-unquoted_string-check', :require => false\n gem 'puppet-lint-undef_in_function-check', :require => false\n gem 'puppet-lint-leading_zero-check', :require => false\n gem 'puppet-lint-trailing_comma-check', :require => false\n gem 'puppet-lint-file_ensure-check', :require => false\n gem 'puppet-lint-version_comparison-check', :require => false\n \n # You can also use the voxpupuli-test gem,\n # which pulls rake, puppet-lint & plugins as dependencies\n gem 'voxpupuli-test', :require => false\nend\n You can then run the lint with:
\n$ bundle install --path vendor/bundle $ bundle exec rake lint\n# And if you want to autofix the detected mistakes\n$ PUPPETLINT_FIX=yes bundle exec rake lint\n Onceover is a toolbox to automate tasks for Puppet control repositories. Among other things, its code quality plugin allows to run syntax checks and invoke\npuppet-lint\n.
In order to use it, update your\nGemfile\n, e.g.:
source ENV['GEM_SOURCE'] || "https://rubygems.org"\n\ngroup :development, :test do\n gem 'voxpupuli-test', :require => false\n \n gem 'onceover', :require => false\n gem 'onceover-codequality', :require => false\nend\n Refresh your bundle and run onceover:
\n$ bundle update $ bundle exec onceover run codequality --no-docs\n Ideally, run this on every commit in a Continuous Integration/Continuous Deployment setup. At Camptocamp, we use a GitLab CI pipeline to check our control repo using Onceover before deploying it with r10k (also running a GitLab CI runner).
\n![\"PuppetMaster](\"https://www.camptocamp.com/wp-content/uploads/puppetmaster_pipeline.png\")
You've checked the quality of your existing code. Good! But what if you're actually maintaining and cleaning code that you don't use anymore? This would be quite the waste of time... At Camptocamp, we've built on\npuppet-lint\nto provide a system to detect unused code and help us clean it up.
This is what the puppet-ghostbuster project is for. Under the hood,\npuppet-ghostbuster\nis a collection of\npuppet-lint\nplugins, distributed in a single\npuppet-ghostbuster\ngem.
These plugins analyze your Puppet code and then connect to your PuppetDB to check if that code is actually used for any known node. It can also check Hiera data for unused keys. Just as previously, you can set it up as a Rake task, but our current setup requires a patch to\npuppet-lint\nin order to whitelist the\npuppet-lint\nchecks activated (the current release of\npuppet-lint\nonly supports blacklisting checks in Rake tasks).
source ENV['GEM_SOURCE'] || "https://rubygems.org"\n\ngroup :development, :test do\n gem 'rake', :require => false\n gem 'puppet-lint', :require => false,\n :git => 'https://github.com/raphink/puppet-lint',\n :ref => '2cac4fb' # Includes patch for whitelisting checks\n gem 'puppet-ghostbuster', :require => false\nend\n You can then set up a Rake task such as this one:
\nPuppetLint::RakeTask.new :ghostbuster do |config|\n config.pattern = ['./site/**/*']\n config.only_checks = [\n 'ghostbuster_classes',\n 'ghostbuster_defines',\n 'ghostbuster_facts',\n 'ghostbuster_files',\n 'ghostbuster_functions',\n 'ghostbuster_hiera_files',\n 'ghostbuster_templates',\n 'ghostbuster_types',\n ]\n config.fail_on_warnings = true\nend\n puppet-ghostbuster\nrequires info to connect to your PuppetDB, so you need to provide the following environment variables:
PUPPETDB_URL\n: URLs to PuppetDB
PUPPETDB_CERT_FILE\n: path to the certificate to use to connect to\nPuppetDB
PUPPETDB_KEY_FILE\n: path to the key to use to connect to PuppetDB
PUPPETDB_CACERT_FILE\n: path to the Puppet CA certificate
HIERA_YAML_PATH\n: path to\nhiera.yaml\nto use
If you don't want to provide certificates and keys, you can connect to the PuppetDB through the unencrypted port 8080, for example by forwarding it through SSH. At Camptocamp, we're automating this setup by using summon as a wrapper to launch the command.
\nWe store the certificates and keys to connect to PuppetDB in gopass, then provide a\nsecrets.yml\nfile like so:
PUPPETDB_URL: https://puppetdb.example.com:8081\nPUPPETDB_CERT_FILE: !var:file path/to/secret:cert\nPUPPETDB_KEY_FILE: !var:file path/to/secret:key\nPUPPETDB_CACERT_FILE: !var:file path/to/secret:cacert\nHIERA_YAML_PATH: ./hiera.yaml\n Which allows to run:
\n$ summon bundle exec rake ghostbuster\n This returns a list of classes, defines, files, templates, etc. that are unused in our code. We can then check these results and clean up our code!
\nDo you have ideas to contribute to\npuppet-ghostbuster\n? Pull requests are welcome! You can also contact us for quotes on Puppet consulting or training!
This post was originally published on https://www.camptocamp.com/actualite/cleaning-up-puppet-code/
\nThis post is also available on DEV.
\n "},{"url":"/posts/configuration-surgery-with-go-structure-tags-12a4/","relativePath":"posts/configuration-surgery-with-go-structure-tags-12a4.md","relativeDir":"posts","base":"configuration-surgery-with-go-structure-tags-12a4.md","name":"configuration-surgery-with-go-structure-tags-12a4","frontmatter":{"title":"Configuration surgery with Go structure tags","stackbit_url_path":"posts/configuration-surgery-with-go-structure-tags-12a4","date":"2020-06-10T20:55:35.728Z","excerpt":"Narcissus is a reflection library letting you edit configuration files in Go","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--vEyenOH7--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://dev-to-uploads.s3.amazonaws.com/i/p9blragy3jsbexp72cs0.jpg","comments_count":0,"positive_reactions_count":5,"tags":["go","opensource","augeas","showdev"],"canonical_url":"https://dev.to/raphink/configuration-surgery-with-go-structure-tags-12a4","template":"post"},"html":"From Docker to Kubernetes, from Consul to Terraform, Go has been used increasingly in system tools these last years.
\nSince most of these system tools manage systems running on Unix systems, one of their core tasks is to deal with files, and configuration files in particular.
\nAugeas is a C library to modify configuration files. It allows to parse files with many different syntax (over 300 by default), modify the configuration using a tree accessed with an XPath-like language, and write back the configuration.
\nIt tries hard to modify only what you mean to, keeping all details (spaces, indentations, new lines, comments) unchanged.
\n![\"Augeas\"](\"https://dev-to-uploads.s3.amazonaws.com/i/xij7ocuklz6agg2rm95h.png\")
Because of its history, Augeas is mainly known in the Puppet world. However, there are also plugins for Ansible, Chef, SaltStack, (R)?ex and more tools… Augeas is also used directly in C libraries such as libvirt and Nut.
\nIn the Puppet world, the Augeasproviders project was created to develop native Puppet types and providers (in Ruby) based on Augeas.
\nThese providers use the Augeas Ruby bindings to draw on Augeas' power, all the while providing a simple interface for users, without the need to know how Augeas works.
\nAt the core of the Augeasproviders project, there is a base provider shipped in the hearculesteam-augeasproviders_core Puppet module, which provides an interface to build more providers, in a declarative way.
\nFor example, you can set the location of the node corresponding to the Puppet resource to manage in the Augeas tree:
\nresource_path do |resource|\n service = resource[:service]\n type = resource[:type]\n mod = resource[:module]\n control_cond = (resource[:control_is_param] == :true) ? "and \n control='# {resource[:control]}'" : ''\n if target == '/etc/pam.conf'\n "$target/*[service='# {service}' and type='# {type}' and module='# {mod}' # {control_cond}]"\n else\n "$target/*[type='# {type}' and module='# {mod}' # {control_cond}]"\n end\nend\n The\ncreate\nand\ndestroy\nmethods, as well as the getters and setters for the Puppet resource properties, can also be described in a similar fashion, making it simpler to develop new providers based on Augeas.
As for many other languages, there are Go bindings for Augeas:
\n\nMuch like the Ruby bindings, the go library lets you manipulate an Augeas handler to query the Augeas tree, modify it, and save it.
\nIn the Go world, structures have optional tags which can be used for parsing and writing to external formats.
\nThis is used to reflect structures as JSON, YAML, XML, or specify library options to manage the structure fields:
\n// Version is an S3 bucket version\ntype Version struct {\n ID uint `\nsql:"AUTO_INCREMENT" gorm:"primary_key" json:"-"\n`\n VersionID string `\ngorm:"index" json:"version_id"\n`\n LastModified time.Time `\njson:"last_modified"\n`\n}\n They are also used to build program interfaces by specifying configuration options:
\ntype config struct {\n Version bool `\nshort:"V" long:"version" description:"Display version."\n`\n Token string `\nshort:"t" long:"token" description:"GitHub token" env:"GITHUB_TOKEN"\n`\n Users []string `\nshort:"u" long:"users" description:"GitHub users to include (comma separated)." env:"GITHUB_USERS" env-delim:","\n`\n Manpage bool `\nshort:"m" long:"manpage" description:"Output manpage."\n`\n}\n The tags above (\nsql\n,\ngorm\n,\njson\n,\nshort\n,\nlong\n,\ndescription\n,\nenv\n,\nenv-delim\n) are used by Go libraries through the Go reflection library to provide dynamic features for structures.
While Hercules is known in Greek mythology for his works —including cleaning the stables of King Augeas—, Narcissus is famous for gazing at his reflection in the water.
\n\nThe Narcissus project is a Go library providing structure tags to manage configuration files with Augeas. It then maps structure tags to the Augeas tree dynamically, allowing you to expose any configuration file (or file stanza) known to Augeas as a Go structure.
\n/etc/group
The Unix\ngroup\nfile is very simple and well-known. It features one group per line, with fields separated by colons:
root:x:0:\ndaemon:x:1:\nbin:x:2:\nsys:x:3:\nadm:x:4:syslog,raphink\n Augeas parses it by storing each group name as a node key in the tree, and exposing each field by its name:
\n$ augtool print /files/etc/group\n/files/etc/group\n/files/etc/group/root\n/files/etc/group/root/password = "x"\n/files/etc/group/root/gid = "0"\n/files/etc/group/daemon\n/files/etc/group/daemon/password = "x"\n/files/etc/group/daemon/gid = "1"\n/files/etc/group/bin\n/files/etc/group/bin/password = "x"\n/files/etc/group/bin/gid = "2"\n/files/etc/group/sys\n/files/etc/group/sys/password = "x"\n/files/etc/group/sys/gid = "3"\n/files/etc/group/adm\n/files/etc/group/adm/password = "x"\n/files/etc/group/adm/gid = "4"\n/files/etc/group/adm/user[1] = "syslog"\n/files/etc/group/adm/user[2] = "raphink"\n Modifying any of these fields and saving the tree will result in an updated\n/etc/group\nfile. Adding new entries in the tree will result in additional entries in\n/etc/group\n, provided the tree is valid for the\nGroup.lns\nAugeas lens.
In our Go code, we can map a\ngroup\nstructure to entries in the\n/etc/group\nfile easily by using the Narcissus package:
import (\n\t"log"\n\n\t"honnef.co/go/augeas"\n\t"github.com/raphink/narcissus"\n)\n\ntype group struct {\n\taugeasPath string\n\tName string `\nnarcissus:".,value-from-label"\n`\n\tPassword string `\nnarcissus:"password"\n`\n\tGID int `\nnarcissus:"gid"\n`\n\tUsers []string `\nnarcissus:"user"\n`\n}\n\nfunc main() {\n\taug, err := augeas.New("/", "", augeas.None)\n\tif err != nil {\n\t\tlog.Fatal("Failed to create Augeas handler")\n\t}\n\tn := narcissus.New(&aug)\n\n\tgroup := &group{\n\t\taugeasPath: "/files/etc/group/docker",\n\t}\n\terr = n.Parse(group)\n\tif err != nil {\n\t\tlog.Fatalf("Failed to retrieve group: %v", err)\n\t}\n\n\tlog.Printf("GID=%v", group.GID)\n\tlog.Printf("Users=%v", strings.Join(group.Users, ","))\n}\n The\naugeasPath\nfield is necessary to store the location of the file in the Augeas tree, in our case\n/files/etc/group/docker\nto manage the\ndocker\ngroup in the file.
Then each structure field is linked to the corresponding node name in the Augeas tree:
\n.,value-from-label\n, where\n.\nrefers to the current node, and\nvalue-from-label\ntells Narcissus how to get the valuepassword\nfor the Password
gid\nfor the GID
user\nfor the Users, parsed as a slice of strings (i.e. the\nuser\nlabel might appear more than once in the Augeas tree)
Note that all fields must be capitalized in order for Go reflection to work.
\nOnce we call the\nParse()\nmethod on the Narcissus handler, the structure is dynamically filled with the values in the Augeas tree, so we can access the gid with\ngroup.GID\nand the users with\ngroup.Users\n.
The main point of the Augeas library is not just to parse, but also to modify configuration files in a versatile way.
\nIn Narcissus, this is done by calling the\nWrite()\nmethod on the Narcissus handler. Narcissus then transforms the structure back to the Augeas tree and saves it.
For example, using the\nPasswdUser\ntype provided by default in the\nnarcissus\npackage:
user := n.NewPasswdUser("raphink")\n\n// Modify UID\nuser.UID = 42\n\nif err := n.Write(user); err != nil {\n log.Fatalf("Failed to save user: %v", err)\n}\n Narcissus comes with a few structures already mapped:
\n/etc/fstab\n, with the\nNewFstab()\nmethod
/etc/hosts\nwith the\nNewHosts()\nmethod
/etc/passwd\nwith\nNewPasswd()\nand\nNewPasswdUser()\nmethods
/etc/services\nwith\nNewServices()\nand\nNewService()\nmethods
Which structures will you map with it? Which tool could benefit from this library?
\nLet me know in the comments!
\nThis post is also available on DEV.
\n "},{"url":"/posts/diffing-puppet-environments-1fno/","relativePath":"posts/diffing-puppet-environments-1fno.md","relativeDir":"posts","base":"diffing-puppet-environments-1fno.md","name":"diffing-puppet-environments-1fno","frontmatter":{"title":"Diffing Puppet Environments","stackbit_url_path":"posts/diffing-puppet-environments-1fno","date":"2020-05-01T14:49:22.209Z","excerpt":"Puppet Catalog Diff helps to visualize the differences between two Puppet environments","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--X2kztkXc--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://www.camptocamp.com/wp-content/uploads/xformations_puppet1-720x400.png.pagespeed.ic.UU2oY1Zlj8.webp","comments_count":0,"positive_reactions_count":5,"tags":["puppet","devops","testing","codequality"],"canonical_url":"https://dev.to/camptocamp-ops/diffing-puppet-environments-1fno","template":"post"},"html":"Puppet is a great tool for configuration managements, allowing to automate hundreds to thousands of nodes at a time in an Infrastructure-as-Code approach.
\nGood practice usually encourages to use multiple environments in a Puppet setup. Usually, critical nodes are pinned to the\nproduction\nenvironment, while less critical nodes can be associated with staging environments.
Using Git along with a Control Repository, code changes are typically produced in feature branches which get turned to Puppet environments. Once features have been tested on said environment, the feature branch can be merged into a staging branch, where the changes will start affecting nodes pinned to that Puppet environment.
\nFinally, once in a while, changes are merged from the staging branch into the production branch, thus affecting all nodes pinned to production.
\nWhile the workflow described is helpful, validating a branch is often a lacking process. Pointing all staging nodes to a feature branch is missing the point entirely, so validation is often done manually, by identifying nodes that may be impacted by the change and running Puppet manually on these nodes, preferably in dry-run mode (\n--noop\n).
When deploying to hundreds of nodes, testing a few is hardly a guaranty that things will go well on all nodes once the branch is merged.
\nFortunately for us, there are tools which can help!
\nIt might be hard to believe as this module is so poorly known, but the Puppet Catalog Diff project was started some 10 years ago by R.I. Pienaar! Adopted by Zack Smith, it was maintained for a few years, but left mainly unmaintained since 2016.
\nAs we've used it for years (and GitHub's octocatalog_diff never fit my need), we've adopted it and you will now find the latest version on our GitHub account:
\n\nPuppet Catalog Diff is a standard Puppet module. You can thus install it using\npuppet module install\n, r10k, or even just git.
$ git clone https://github.com/camptocamp/puppet-catalog-diff.git /etc/puppetlabs/code/modules/catalog_diff\n As its name implies, Puppet Catalog Diff allows to perform diffs between Puppet catalogs.
\nThe module provides three Puppet faces:
\npuppet catalog seed\ngenerates catalogs from a Puppet Master (or PuppetDB)
puppet catalog pull\nwraps around the\nseed\nface to retrieve catalogs from two environments for each node
puppet catalog diff\nanalyzes multiple catalogs and returns the differences per node
To get started, you can diff local catalogs (in\n.yaml\n,\n.pson\n, or\n.yaml\nformats):
$ puppet catalog diff catalog1.pson catalog2.pson\n will return the differences between the two catalogs.
\nMost often, you will want to use Puppet Catalog Diff to retrieve catalogs from Puppet Masters.
\nEverything the Puppet world uses OpenSSL for authentication. Setting up Puppet Catalog Diff will thus require an OpenSSL certificate. This can be any certificate signed by the Puppet CA. For example, you can use the \npuppetserver ca\ncommand to generate a certificate:
$ puppetserver ca generate --certname catalog-diff\n Retrieve the key and certificate.
\nBy default, Puppet Masters only deliver catalogs for the nodes requesting them. This is set up in the \nauth.conf\n configuration file, with a rule like:
{\n # Allow nodes to retrieve their own catalog\n match-request: {\n path: "^/puppet/v3/catalog/([^/]+)$"\n type: regex\n method: [get, post]\n }\n allow: "$1"\n sort-order: 500\n name: "puppetlabs catalog"\n},\n You can deploy this rule using the\npuppet_authorization::rule\ndefined type from the puppet_authorization Puppet module.
To allow the\ncatalog-diff\ncertificate to access get any catalog from the Puppet Master, we can modify that rule:
{\n # Allow nodes to retrieve their own catalog\n match-request: {\n path: "^/puppet/v3/catalog/([^/]+)$"\n type: regex\n method: [get, post]\n }\n allow: ["$1","catalog-diff"]\n sort-order: 500\n name: "puppetlabs catalog"\n},\n Even better yet, we can add a certificate extension to the catalog diff certificate, e.g.\npp_authorization: catalog\nand allow this extension in\nauth.conf\n:
{\n # Allow nodes to retrieve their own catalog\n match-request: {\n path: "^/puppet/v3/catalog/([^/]+)$"\n type: regex\n method: [get, post]\n }\n allow: [\n "$1",\n {\n extensions: {\n pp_authorization: "catalog"\n }\n }\n ]\n sort-order: 500\n name: "puppetlabs catalog"\n},\n When comparing environments, Puppet Catalog Diff will connect to one or multiple Puppet Masters and get catalogs for each node.
\nAs you may have many nodes to test, it is easier to get the list of nodes to analyze from the PuppetDB. This can be achieved with the\n--use_puppetdb\n, along with\n--filter_old_env\n. This will select all the active nodes in the Puppet associated with the first environment.
For example, if we run:
\n$ puppet catalog diff \\\n puppet.example.com/production \\\n puppet.example.com/staging \\\n --use_puppetdb --filter_old_env\n Note (2020-05-07): Since the release of Puppet Catalog Diff 2.0.0,\n`--usepuppetdbis now deprecated and--filteroldenv`\nis the default._
Puppet Catalog Diff will connect to the PuppetDB, get all the active nodes from the\nproduction\nenvironment, and then for each of them, retrieve a catalog for the node from:
production\nenvironment on the\npuppet.example.com\nPuppet Masterstaging\nenvironment on the\npuppet.example.com\nPuppet MasterIt will then compute differences between each pair of catalogs and output them.
\nOne type of check that is very necessary is testing changes between two versions of Puppet Master installations. Puppet Catalog Diff allows you to specify different masters for the two environments to compare, so you can use the following command to compare catalogs from two Puppet Masters on the same Puppet environment (provided the environment is deployed to both masters):
\n$ puppet catalog diff \\\n puppet5.example.com/production \\\n puppet6.example.com/production \\\n --use_puppetdb --filter_old_env\n Retrieving and comparing catalogs can be resource-consuming. Very often, you will want to diff a new environment (staging or feature) against a more stable one. Since we can get the nodes associated to the stable environment from PuppetDB, we might as well get the cached catalogs from PuppetDB for this branch, too. This is possible using the\n--old_catalog_from_puppetdb\nflag:
$ puppet catalog diff \\\n puppet.example.com/production \\\n puppet.example.com/staging \\\n --use_puppetdb --filter_old_env --old_catalog_from_puppetdb\n Catalogs from will retrieved from PuppetDB for the\nproduction\nenvironment, and from the Puppet Master for the\nstaging\nenvironment.
Several options are available to tune the diff output:
\n--show_resource_diff\nwill show the details of how each resource was modified
--content_diff\nwill generate a separate content diff for file contents, in addition to the parameters diff
--changed_depth 1000\nsets the number of nodes to display at the end of the diff, sorted by amount of diffs
Puppet provides a special variable named\n$trusted\nand called Trusted Facts. This variable contains information from the Puppet certificate. This allows the Puppet Master to get informations, such as the certname or the certificate extensions, and be sure that they could not be falsified.
However, using these trusted facts in your Puppet code (or Hiera hierarchy) breaks compilation with Puppet Catalog Diff, since the catalog diff's certificate does not contain these trusted variables.
\nIf you are using Puppet 6.3 or up on your Puppet Master, you can make use of the new certless catalog API to bypass this restriction.
\nSince this uses a different API endpoint, we need to set up\nauth.conf\nfor it, for example:
{\n # Allow nodes to retrieve their own catalog\n match-request: {\n path: "^/puppet/v4/catalog"\n type: regex\n method: [post]\n }\n allow: [\n {\n extensions: {\n pp_authorization: "catalog"\n }\n }\n ]\n sort-order: 500\n name: "puppetlabs certless catalog"\n},\n The\n--certless\nflag will tell Puppet Catalog Diff to use the new certless catalog API in place of the standard one.
For example, you can retrieve the old catalogs from PuppetDB and the new catalogs from the certless catalog API:
\n$ puppet catalog diff \\\n puppet.example.com/production \\\n puppet.example.com/staging \\\n --use_puppetdb --filter_old_env \\\n --old_catalog_from_puppetdb --certless\n If you are using a Continuous Integration platform, you can get advantage of it by integrating your Puppet control repository into it with Puppet Catalog Diff.
\nWhile general Code Quality tasks can be launched in a pipeline before deploying the code, Puppet Catalog Diff is typically a task that can be lauched in a merge request.
\nFor example, you can launch the following command in a GitLab CI job:
\n$ puppet catalog diff \\\n puppet.example.com/${CI_MERGE_REQUEST_TARGET_BRANCH_NAME} \\\n puppet.example.com/${CI_MERGE_REQUEST_SOURCE_BRANCH_NAME} \\\n --show_resource_diff --content_diff --changed_depth 1000 \\\n --use_puppetdb --filter_old_env --old_catalog_from_puppetdb \\\n --certless --threads 4 \\\n --output_report /srv/catalog-diff/mr_${CI_MERGE_REQUEST_IID}_${CI_JOB_ID}.json\n The\n--output_report\noption saves the output as a JSON document, which can be used later on.
Puppet Catalog Diff compares Puppet catalogs. However, catalog changes do not account for all changes in a Puppet agent run. Plugins can play a role, too.
\nIf your change involves a change in agent-side plugins (facts, types & providers, augeas lenses), Puppet Catalog Diff won't allow you to predict the result of these changes.
\nChanges in Puppet code sometimes generate a lot of diff, which can be hard to parse in text form.
\nThe Puppet Catalog Diff Viewer project allows to visualize Puppet Catalog Diff reports (as generated by\n--output_report\noption) in a Web UI.
This interface is currently read-only, with no persistence.
\n![\"Puppet](\"https://dev-to-uploads.s3.amazonaws.com/i/b73sm2n5ie2yip4hh427.png\")
Let me know how you use Puppet Catalog Diff, and, as usual, we welcome Pull Request!
\nThis post is also available on DEV.
\n "},{"url":"/posts/enhance-and-colorize-old-pictures-5c9g/","relativePath":"posts/enhance-and-colorize-old-pictures-5c9g.md","relativeDir":"posts","base":"enhance-and-colorize-old-pictures-5c9g.md","name":"enhance-and-colorize-old-pictures-5c9g","frontmatter":{"title":"Enhance, Colorize, and Animate Old Pictures","stackbit_url_path":"posts/enhance-and-colorize-old-pictures-5c9g","date":"2020-06-29T16:45:38.653Z","excerpt":"MyHeritage in Color allows to fine-tune automatically colorized and enhanced photographs","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--PKPnYlZ7--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://dev-to-uploads.s3.amazonaws.com/i/b78xjnsl62d9by6bkyoh.jpg","comments_count":0,"positive_reactions_count":11,"tags":["tutorial","ai","machinelearning","photography"],"canonical_url":"https://dev.to/raphink/enhance-and-colorize-old-pictures-5c9g","template":"post"},"html":"Over the last 2 years, Machine Learning has brought impressive breakthrough to image processing techniques. In particular, photography colorization has seen amazing progress, thanks mainly to the work of two developers: Jason Antic and Dana Kelley.
\nTheir model is so precise that MyHeritage hired them to include a colorization tool directly on their website. As a result, you can colorize pictures for free at https://www.myheritage.fr/incolor. Paid MyHeritage members are not limited in the number colorizations, and don't get the MyHeritage watermark on the result pictures.
\nAdditionally, MyHeritage recently added a new AI-based feature to this tool, by allowing users to enhance faces in their pictures:
\n\nUsing the tool from your MyHeritage picture collection is extremely simple. You just need to upload a picture and use the \"enhance\" and \"colorize\" buttons one after the other.
\n![\"Colorization](\"https://dev-to-uploads.s3.amazonaws.com/i/wt2uwlf0t0dhu2w53w91.png\")
In the last few weeks, I've seen lots of people using MyHeritage In Color on their pictures, and they're usually quite content with the result. However, most of them have no idea they could get even better results by tuning the rendering.
\nOnce a picture is colorized, a gear icon appears to let you fine-tune the result:
\n![\"Tuning](\"https://dev-to-uploads.s3.amazonaws.com/i/4prlyuwyikgo0qhi9v69.png\")
There are 4 parameters which can be tuned using this icon:
\nGenerally speaking, this is what I have found from colorizing hundreds of pictures, essentially portraits of people:
\nHere is an example:
\n![\"Default](\"https://dev-to-uploads.s3.amazonaws.com/i/of5je7h1ou8nd48dtugh.png\")
This colorized (and enhanced) picture doesn't look bad, but the clothes have lots of purple spots all over them. Switching to the alternative model reduces this, without affecting the faces too much:
\n![\"Alternative](\"https://dev-to-uploads.s3.amazonaws.com/i/epmwc8x4r4zp90nu6jju.png\")
There's still some weird zones though, so I want to try and reduce these by increasing the rendering factor. After playing around, I ended up with a factor of 64 for that picture:
\n![\"Render](\"https://dev-to-uploads.s3.amazonaws.com/i/sqv6a9c7s7rrb2gtb9l1.png\")
However, the colors are getting slightly dim now, so I've increased the saturation to 1.2 to counteract that:
\n![\"Saturation\"](\"https://dev-to-uploads.s3.amazonaws.com/i/l6ttuj25eh7jxtzolp28.png\")
Granted, the results on this picture are far from perfect, as it is hard to colorize (which is why I chose it!).
\nWhen tuning a colorization, every time you modify a parameter, you can preview the changes, and then choose to save them.
\nIn a very practical fashion, MyHeritage lets you click on the colorized picture to compare with the previously saved version.
\nHowever, it doesn't save the results for each combination, so when you play around with parameters, you can end up waiting a long time to go back to previous combinations.
\nFor this reason, I use A/B testing when colorizing. My process is:
\nThe drawback of this method is that you need to re-open the tuning interface every time, but in the end, you'll save lots of rendering time and you know you're saving the best version you could get.
\nWhile the settings in MyHeritage in Color clearly help, they can't —yet— get you to a perfect result most of the time.
\nIn many cases, some parameters will have better results on people, while others will improve objects or the background.
\nOne thing I've started doing is downloading multiple versions of the colorization, importing them all as layers in Gimp, and cutting them so as to get the best result in every zone.
\nIn other cases, the tint on some objects may not be perfect despite trying to tune the engine. In these cases, you can duplicate these zones in Gimp and tune their tint/saturation/hue individually until you get the result you want.
\nFace enhancing can lead to very impressive results by recreating realistic faces that match the shadow of faces in the blur pictures.
\nFor example, in the picture I used before, some of the children's faces look amazing:
\n![\"Face](\"https://dev-to-uploads.s3.amazonaws.com/i/2svy7z72gnqo6cl90oql.jpg\")
Yes, the original is on the left!
\nAt times however, little dots or other issues with the original picture can lead the face enhancer astray and generate some very disturbing results. Here for example, there's a line on the face in the original picture, which gets \"interpreted\" as a scar:
\n![\"Scar](\"https://dev-to-uploads.s3.amazonaws.com/i/i8d5qjc1ysxlegxooylj.jpg\")
And in this one, the left eye looks too dark in the original picture, leading to a strange color dissymetry:
\n![\"Eye](\"https://dev-to-uploads.s3.amazonaws.com/i/dbzchaqre1c8eyy3zc6e.jpg\")
Fortunately, these aren't too hard to fix. In the case of the \"scar\", erasing it (using Gimp with simple editing methods such as the stamp tool) fixed the issue:
\n![\"Fixed](\"https://dev-to-uploads.s3.amazonaws.com/i/b78xjnsl62d9by6bkyoh.jpg\")
The eyes weren't much harder to fix. Since the face is straight, I just copied the right pupil (just the pupil, not the entire eye) and pasted it on the left eye. The difference is hardly noticeable in the B&W picture, but it's enough for the AI algorithm to pick it up properly:
\n![\"Fixed](\"https://dev-to-uploads.s3.amazonaws.com/i/2xxhxugvmo4wqnxdj2qa.jpg\")
As of February 2021, MyHeritage now allows to animate portraits that were enhanced.
\nFrom the picture view, simply click the animation button and choose the face to animate. You can choose from 10 different animations. If the face was colorized, the animation will honor it.
\n\nHave you colorized pictures with DeOldify or MyHeritage in Color? Have you found useful ways to get good results? Share them in the comments!
\nThis post is also available on DEV.
\n "},{"url":"/posts/git-markdown-to-write-a-novel-ag6/","relativePath":"posts/git-markdown-to-write-a-novel-ag6.md","relativeDir":"posts","base":"git-markdown-to-write-a-novel-ag6.md","name":"git-markdown-to-write-a-novel-ag6","frontmatter":{"title":"Git & Markdown to write a novel","stackbit_url_path":"posts/git-markdown-to-write-a-novel-ag6","date":"2020-05-09T06:20:31.422Z","excerpt":"Using Git and Markdown to write a novel","thumb_img_path":null,"comments_count":0,"positive_reactions_count":1,"tags":["documentation","markdown","writing"],"canonical_url":"https://dev.to/raphink/git-markdown-to-write-a-novel-ag6","template":"post"},"html":"This year, I started writing a historical novel about a branch of my family. I quickly realized I needed some tooling to organize my data: what I know about the characters, the places, a general timeline, etc.
\nI looked for software to do that and found that the reference is Scrivener. However interesting it looked, I'd rather use Open Source software whenever possible, so I started using Manuskript, an Open Source software for writers similar to Scrivener.
\n![\"Manuskript](\"https://dev-to-uploads.s3.amazonaws.com/i/fj7xlois1xl9d7qtzyjl.png\")
I laid out some chapters, characters, plots… then I realized the saving format was binary. It's actually a Zip archive which contains all the information in flat files:
\n 1 2020-05-09 08:05 MANUSKRIPT\n 137 2020-05-09 08:05 infos.txt\n 0 2020-05-09 08:05 summary.txt\n 46 2020-05-09 08:05 status.txt\n 147 2020-05-09 08:05 labels.txt\n 284 2020-05-09 08:05 characters/0-Samuel_L-on.txt\n 281 2020-05-09 08:05 characters/1-L-on_Grunberg.txt\n 261 2020-05-09 08:05 characters/2-Adolphe_Grunberg.txt\n 309 2020-05-09 08:05 characters/3-Elisabeth_Rau.txt\n 112 2020-05-09 08:05 characters/4-Victor_Grunberg.txt\n 109 2020-05-09 08:05 characters/5-Maria_Schorr.txt\n 224 2020-05-09 08:05 characters/6-Fred_Grunberg.txt\n 125 2020-05-09 08:05 characters/7-Colonel_de_Villebois-Mareuil.txt\n 107 2020-05-09 08:05 characters/8-Said_Pacha.txt\n 109 2020-05-09 08:05 characters/9-Isaac_Aghion.txt\n 92 2020-05-09 08:05 outline/00-Setup.md\n 322 2020-05-09 08:05 outline/01-Les_joyaux_-gyptiens/folder.txt\n 3753 2020-05-09 08:05 outline/01-Les_joyaux_-gyptiens/0-Une_rencontre_inattendue.md\n 1706 2020-05-09 08:05 outline/01-Les_joyaux_-gyptiens/1-Office_de_chabbat.md\n 715 2020-05-09 08:05 outline/01-Les_joyaux_-gyptiens/2-Un_d-ner_chez_Isaac.md\n 482 2020-05-09 08:05 outline/01-Les_joyaux_-gyptiens/3-Proc-s_1.md\n 166 2020-05-09 08:05 outline/01-Les_joyaux_-gyptiens/4-Sc-ne_5.md\n 83 2020-05-09 08:05 outline/02-Paris/folder.txt\n 102 2020-05-09 08:05 outline/02-Paris/0-Sc-ne_1.md\n 97 2020-05-09 08:05 outline/03-Etudes_des_enfants/folder.txt\n 103 2020-05-09 08:05 outline/03-Etudes_des_enfants/0-Sc-ne_1.md\n 103 2020-05-09 08:05 outline/03-Etudes_des_enfants/1-Sc-ne_2.md\n 95 2020-05-09 08:05 outline/04-Premiers_emplois/folder.txt\n 103 2020-05-09 08:05 outline/04-Premiers_emplois/0-Sc-ne_1.md\n 96 2020-05-09 08:05 outline/05-Vers_le_Transvaal/folder.txt\n 103 2020-05-09 08:05 outline/05-Vers_le_Transvaal/0-Sc-ne_1.md\n 334 2020-05-09 08:05 outline/21-Epilogue.md\n 40074 2020-05-09 08:05 revisions.xml\n 275 2020-05-09 08:05 world.opml\n 1669 2020-05-09 08:05 plots.xml\n 2499 2020-05-09 08:05 settings.txt\n This is rather good news, and there's ways to automate PDF creation from this within the software, good news again!
\nHowever, some things were problematic to me:
\nSo I decided I would do everything in Git + Markdown, without the help of a third-party application.
\nI'm currently storing the chapters in their own directories, with numbered Markdown files:
\nchapitres/\n├── 01_les_joyaux_egyptiens\n│ ├── 00_titre.md\n�� ├── 01_une_rencontre_innatendue.md\n│ ├── 02_office_de_chabbat.md\n│ ├── 03_brody.md\n│ ├── 04_chabbat_2.md\n│ ├── 05_diner_chez_isaac.md\n│ └── 06_audience.md\n├── 02_les_enfants_grunberg\n│ ├── 00_titre.md\n│ ├── 01_leon.md\n│ ├── 02_retour.md\n│ ├── 03_coffre.md\n│ ├── 04_burtaux.md\n│ ├── 05_article.md\n│ └── 06_article2.md\n├── 03_annees_noires\n│ └── 00_titre.md\n├── 04_etudes\n│ └── 00_titre.md\n├── 05_premier_travail\n│ └── 00_titre.md\n├── 06_le_creusot\n│ └── 00_titre.md\n└── 21_epilogue.md\n The characters documentation is stored in Markdown files as well, and links can be made between them when necessary:
\npersonnages/\n├── Adolphe_Grunberg.md\n├── Charlotte_Grunberg.md\n├── Elisabeth_Rau.md\n├── Emilie_Grunberg.md\n├── Felix_Zottier.md\n├── Frederic_Grunberg.md\n├── Isaac_Aghion.md\n├── Jacques_Grunberg.md\n├── Leon_Grunberg.md\n├── Lucie_Leon.md\n├── Marc_Leon.md\n├── Mirel_Schorr.md\n├── Paul_Grunberg.md\n└── Samuel_Leon.md\n Same goes for places:
\nlieux/\n├── Alexandrie.md\n├── Brody.md\n├── Dubno.md\n├── Grunberg_Boulogne.md\n├── Paris.md\n├── Petite_Jonchere.md\n└── Vienne.md\n Links can easily be made between these documentation files:
\n# Mirel Schorr\n\n\n## État civil\n\n* Prénoms : Mirel, dite Maria\n* Nom : Grünberg\n* Nom de naissance : Schorr\n\n\n## Portrait\n\nInconnu\n\n\n## Description physique\n\nInconnu\n\n\n\n## Evénements\n\n\n* ca 1800 : ° à [Dubno](../lieux/Dubno.md) (Russie)\n* Enfance à [Brody](../lieux/Brody.md) (Autriche)\n avec ses parents [Schachne](Schachne_Schorr.md)\n et [Sarah](Sarah_Bick.md)\n et ses frères [Naphtali](Naphtali_Mendel_Schorr.md)\n et [Osias](Osias_Heschel_Schorr.md)\n\n* 1821: Mariage à [Brody](../lieux/Brody.md)\n\n* 1870 : habite à [Vienne](../lieux/Vienne.md) (testament [Adolphe](Adolphe_Grunberg.md))\n\n* 1877 : habite à [Paris](../lieux/Paris.md) (+)\n* 1877 : + à [Vienne](../lieux/Vienne.md)\n\n\n## Habillement\n\nVoir sa garde-robe en 1853 dans l'inventaire de [Victor](Victor_Grunberg.md)\n Manuskript provides stats about the number of words in a section. I've added a Make target for that:
\nstats:\n\tfind chapitres/ -name "*.md" -not -name '00_titre.md' -print0 | sort -z | xargs -0 wc -w\n Caveat: it also lists the words in the comments…
\nI'm using Pandoc to build the project with my own LaTeX template.
\n%.md:\n\tcat meta.md > $@\n\tfind chapitres/ -name "*.md" -print0 | sort -z | xargs -0 cat >> $@\n\n%.tex: %.md\n\tpandoc --pdf-engine lualatex --template extended.tex \\\n\t\t --variable numbersections --toc --variable toc-depth=2 \\\n\t\t --variable documentclass=memoir --variable fontsize=12pt \\\n\t\t --filter pandoc-citeproc \\\n\t\t --verbose \\\n\t\t $< -o $@\n\n%.pdf: %.tex\n\tOSFONTDIR=$(FONTSDIR) lualatex $<\n\tmakeindex $*.idx\n\tOSFONTDIR=$(FONTSDIR) lualatex $<\n The novel project can be found in this GitHub repository:
\n\nThis post is also available on DEV.
\n "},{"url":"/posts/how-to-allow-dynamic-terraform-provider-configuration-20ik/","relativePath":"posts/how-to-allow-dynamic-terraform-provider-configuration-20ik.md","relativeDir":"posts","base":"how-to-allow-dynamic-terraform-provider-configuration-20ik.md","name":"how-to-allow-dynamic-terraform-provider-configuration-20ik","frontmatter":{"title":"How to allow dynamic Terraform Provider Configuration","stackbit_url_path":"posts/how-to-allow-dynamic-terraform-provider-configuration-20ik","date":"2021-05-11T11:47:57.105Z","excerpt":"Terraform providers can be dynamically configured using other resource attributes if their code allows for it","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--rch8h5M6--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bjkdnrg8gmbjiazvn8l8.jpg","comments_count":0,"positive_reactions_count":7,"tags":["terraform","devops","go","cfgmgmt"],"canonical_url":"https://dev.to/camptocamp-ops/how-to-allow-dynamic-terraform-provider-configuration-20ik","template":"post"},"html":"Terraform relies heavily on the concept of providers, a base brick which consists of Go plugins enabling the communication with an API.
\nEach provider gives access to one or more resource types, and these resources then manage objects on the target API.
\nMost of the time, a provider's configuration is static, e.g.
\nprovider "aws" {\n region = "us-east-1"\n}\n However, in some cases, it is useful to configure a provider dynamically, using the attribute values from other resources as input for the provider's configuration.
\nI'll use the example of the Argo CD provider. In a single Terraform run, we would like to:
\nOur code will look like this:
\n# Install Kubernetes & Argo CD using a local module\n# (from https://devops-stack.io)\nmodule "cluster" {\n source = "git::https://github.com/camptocamp/devops-stack.git//modules/k3s/docker?ref=master"\n\n cluster_name = "default"\n node_count = 1\n}\n\n# /!\\ Setup the Argo CD provider dynamically\n# based on the cluster module's output\nprovider "argocd" {\n server_addr = module.cluster.argocd_server\n auth_token = module.cluster.argocd_auth_token\n insecure = true\n grpc_web = true\n}\n\n# Deploy an Argo CD resource using the provider\nresource "argocd_project" "demo_app" {\n metadata {\n name = "demo-app"\n namespace = "argocd"\n }\n\n spec {\n description = "Demo application project"\n source_repos = ["*"]\n\n destination {\n server = "https://kubernetes.default.svc"\n namespace = "default"\n }\n\n orphaned_resources {\n warn = true\n }\n }\n\n depends_on = [ module.cluster ]\n}\n This requires to configure Argo CD dynamically, using the output of the Kubernetes cluster's resources.
\nProviders are initialized early in a Terraform run, as their initialization is required to compute the graph which defines in which order the resources are applied.
\nThis means it is actually not possible to make a provider initialize after a secondary resource is created.
\nOfficially, the story stops here, and Terraform has a bug report to track the feature allowing to dynamically configure providers.
\nSo… it's game over then? 🎮 👾\nNot really!
\nWhen a provider is configured in Terraform, it triggers a configuration function:
\nfunc Provider() *schema.Provider {\n return &schema.Provider{\n ConfigureFunc: func(d *schema.ResourceData) (interface{}, error) {\n // Create someObject\n return someObject, nil\n }\n }\n}\n This\nConfigureFunc\nmethod is usually used to create a static client for the target API. In the Argo CD provider for example, it returns a\nServerInterface\nstructure, with pointers to several clients, instantiated from the provider parameters:
type ServerInterface struct { \n ApiClient *apiclient.Client \n ApplicationClient *application.ApplicationServiceClient \n ClusterClient *cluster.ClusterServiceClient \n ProjectClient *project.ProjectServiceClient \n RepositoryClient *repository.RepositoryServiceClient \n RepoCredsClient *repocreds.RepoCredsServiceClient \n ServerVersion *semver.Version \n ServerVersionMessage *version.VersionMessage \n}\n The return statement from the\nConfigureFunc\neventually looks like this:
return ServerInterface{ \n &apiClient, \n &applicationClient, \n &clusterClient, \n &projectClient, \n &repositoryClient, \n &repoCredsClient, \n serverVersion, \n serverVersionMessage}, err\n Let's add a new field to the\nServerInterface\nto store the pointer to the provider's\nResourceData\nobject, which gives access to the provider's parameters:
type ServerInterface struct { \n ApiClient *apiclient.Client \n ApplicationClient *application.ApplicationServiceClient \n ClusterClient *cluster.ClusterServiceClient \n ProjectClient *project.ProjectServiceClient \n RepositoryClient *repository.RepositoryServiceClient \n RepoCredsClient *repocreds.RepoCredsServiceClient \n ServerVersion *semver.Version \n ServerVersionMessage *version.VersionMessage \n ProviderData *schema.ResourceData \n}\n Now in the\nConfigureFunc\n, we'll instantiate the\nServerInterface\n, providing only the\nProviderData\npointer. The first resource that needs to use the provider will then instantiate the clients, when the provider parameters are available. We'll need the\nConfigureFunc\nmethod to return a pointer to a\nServerInterface\n, so we can later cache the clients and avoid recreating them for every resource:
ConfigureFunc: func(d *schema.ResourceData) (interface{}, error) { \n server := ServerInterface{ProviderData: d} \n return &server, nil \n},\n Now we need to actually initialize the clients in each resource.
\nEach resource method gets the interface returned by the\nConfigureFunc\nfunction as an empty interface parameter, usually called\nmeta\n:
func resourceArgoCDProjectCreate(ctx context.Context, d *schema.ResourceData, meta interface{}) diag.Diagnostics {\n These methods currently simply cast the\nmeta\nparameter as a\nServerInterface\nstructure and use the pre-initialized clients:
server := meta.(ServerInterface)\n We now need to cast\nmeta\nas a pointer to a\nServerInterface\nstructure instead (since we'll need to modify the clients from within the resources), and initialize the clients:
server := meta.(*ServerInterface) \nif err := server.initClients(); err != nil { \n return []diag.Diagnostic{ \n diag.Diagnostic{ \n Severity: diag.Error, \n Summary: fmt.Sprintf("Failed to init clients"), \n Detail: err.Error(), \n }, \n } \n}\n The\ninitClients()\nmethod of the\nServerInterface\nstructure will be called, allowing to set up the clients using the current provider parameters.
In the\nServerInterface# initClients()\nmethod, we want to make sure we reuse existing clients. This is rather simple, since each client is stored as a pointer in the structure, so it defaults to\nnil\n:
func (p *ServerInterface) initClients() error { \n d := p.ProviderData \n \n if p.ApiClient == nil { \n apiClient, err := initApiClient(d) \n if err != nil { \n return err \n } \n p.ApiClient = &apiClient \n }\n\n // etc for all clients\n\n return nil\n}\n That's it, we're done. With these modifications,\nterraform plan\nnow works. The resources get applied in the proper order, and the outputs from the\ncluster\nmodule get properly passed as configuration to the Argo CD clients.
This post is also available on DEV.
\n "},{"url":"/posts/immutability-loose-coupling-a-match-made-in-heaven-37kl/","relativePath":"posts/immutability-loose-coupling-a-match-made-in-heaven-37kl.md","relativeDir":"posts","base":"immutability-loose-coupling-a-match-made-in-heaven-37kl.md","name":"immutability-loose-coupling-a-match-made-in-heaven-37kl","frontmatter":{"title":"Immutability & loose coupling: a match made in heaven","stackbit_url_path":"posts/immutability-loose-coupling-a-match-made-in-heaven-37kl","date":"2021-03-18T07:31:29.616Z","excerpt":"Decoupling in container orchestration enables immutable infrastructure workflows.","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--otN7Kjk1--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zptftzr4jxlblx11pf8n.jpg","comments_count":0,"positive_reactions_count":1,"tags":["devops","containers","immutability","architecture"],"canonical_url":"https://www.camptocamp.com/en/news-events/immutability-and-loose-coupling-a-match-made-in-heaven","template":"post"},"html":"When it comes to infrastructure and deployment automation, two opposite approaches share the podium: mutable vs immutable management.
\nMutable systems usually have a long life cycle, typically in the order\nof weeks to years. As their requirements change (new files,\nconfigurations, users, packages, etc.), the systems are modified to\nmatch a new target state. When left unmanaged, mutable systems tend to\ndrift away from their target state, in a divergent dynamic.
\n![\"Convergence](\"https://dev-to-uploads.s3.amazonaws.com/uploads/articles/11trbxkb1ev9goye0q2k.png\")
Automating mutable systems is often referred to as Configuration Management, and leverages tools such as Cfengine, Puppet, Chef, or Ansible. This tooling uses principles based on the concepts of target state, idempotence, and somewhat related to Mark Burgess’ Promise Theory. Configuration Management aims to make the system convergent, by running a tool on a regular basis, in order to resynchronize the system with its target state. Some of these tools (e.g. mgmt) also attempt to reach congruence by adopting a reactive approach, triggering corrective actions on events.
\nIn an immutable system, any change requires a new deployment. Whether it be a change in configuration, new files, or new users, immutability demands that the system be destroyed and rebuilt from scratch.
\n![\"Trash](\"https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hl5ewtyqpo5offh4p9jh.png\")
An immutable approach can greatly simplify deployments. Avoiding convergence, immutable systems rely on artifacts that are built once, and can be deployed multiple times. These artifacts can increase trust in the ability to rebuild the system from scratch if necessary. It can also ease scalability, since the artifact can be precisely duplicated.
\nHowever, immutability comes with a high cost: in order to be done properly, it must be strict. Any change to the state involves a complete replacement of the artifact. A lack of abiding to that rule results in a convergent system (at best), which cannot be managed in an immutable manner.
\nIf immutable systems are easier to maintain, what are the reasons for not using them? The system’s complexity is probably the main justification. A traditional mutable system features multiple layers (Operating System, Middleware, Applications) that are usually strongly coupled. For example, the flavor and version of the Operating System define which version of a Middleware (e.g. Tomcat, Apache) is available for installation. In turn, the Middleware version defines which libraries are available for the Application. On most Unix-based systems, shared libraries are at the root of strong links between software versions, based on the underlying ABIs required to run them.
\nIf such strongly coupled systems are to be managed in an immutable manner, then the whole system is the immutable artifact. In the majority of organizations, this implies managing tens to thousands of artifacts, and rebuilding them from scratch on a regular basis. Such complexity is too much of a cost.
\nEnters decoupling technologies: Over the years, new technologies have surfaced, which allow to decouple system components and ease their management in an immutable manner.
\nWith the rise of virtualization in the early years of the 21st century, it became easier to decouple the hardware from the Operating System. You could now size virtual machines as precisely as desired, in terms of CPU, memory, or disk, without adding or replacing any physical device.
\nThis unlocked access to a first level of automated immutability, using Virtual Machine images as the immutable artifact. Image generators (such as Hashicorp Packer) appeared, easing the generation of VM images.
\nProvided the whole target state —including the OS, middleware and application itself— is built into the VM image, an immutable workflow can be used to manage it. In this case, whenever a change is required, the whole image needs to be rebuilt and redeployed to new VMs.
\n![\"Golden](\"https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0gu5n8xhs57jvxnkn3ay.jpg\")
\n For a time, this deployment could not be easily automated, as physical nodes still needed to be manually picked before deploying the VM images. Infrastructure as a Service (IaaS), often referred to as “Cloud”, changed that.
\nIaaS provided APIs on top of virtualization, and the IaaS system (e.g. AWS EC2, OpenStack) would often pick the physical hypervisor node itself, making it possible to fully automate the deployment of new artifacts (such as VM images).
\nOne important problem remained to achieve immutable infrastructure in most situations: the complexity of the artifact itself.
\nWhen setting up applications on existing systems, several issues often arise, among which are: packaging, configuration, and dependencies.
\nFor years, developers and systems engineers tried to solve the problem of application packaging and deployment using all kinds of package managers, from deb/rpm to homemade systems. This often failed, because these packages didn’t allow for multiple instantiation of the application, were not easy to configure, and were too tightly coupled to the rest of the system.
\nDocker containers provided a unified way of packaging applications, in the form of OCI images, a rather unified way of configuring them (using environment variables or mounted files, in the 12 factor app fashion).
\nBut mainly, containers provided an abstraction level, a decoupling from the system. With containers, it doesn’t matter anymore which OS is running the container engine. Developers can now choose to run any version of Tomcat or Apache, on any node with a container engine. As a corollary, they can also run any combination of Middleware and Applications, regardless of the libraries provided on the underlying system.
\nFurthermore, containers were made to be managed in an immutable manner, using OCI images as immutable artifacts. Every time a container needs to be modified, it requires the creation of a new container from a new image.
\nThe benefits are huge. With this decoupling of the Operating System from the Middleware and Applications, the monolithic immutable artifact that was previously managed as a VM image can now be broken down into many pieces: the application is now an immutable artifact, and so are the middleware components as well.
\nEven better: since all the components running on the machines are now immutable, the machines themselves have now become totally neutral; all they require is a container engine in order to run containers.
\nOne thing can still make a node a snowflake: manually deployed containers, using tools such as Docker or Docker-compose.
\nWhat IaaS did for VMs, Container orchestrators now do for containers: they provide an API to orchestrate the dynamic deployment of containers on a cluster of nodes.
\n![\"Container](\"https://dev-to-uploads.s3.amazonaws.com/uploads/articles/c4m11g0j9tcebfi9zlic.png\")
\n The nodes are thus totally neutral now. As a result, their templates are greatly simplified, and they can now easily be managed in an immutable way. This new paradigm opened the way to new Operating Systems specialized for container orchestration, such as CoreOS or RancherOS, whose life cycles are meant to be managed with an immutable workflow.
\nNow that we have a full Immutable System, does it solve the problem of convergence?
\nImmutable artifacts in themselves are not supposed to evolve, but their target state does evolve. In this regard, the situation with containers is similar to that of Golden Images for Virtual Machines: though the artifact is immutable, it can easily be used as a template leading to an unmaintained, divergent system.
\nThere is thus still a need for convergence tools in the container world. However, this is not because the objects themselves drift from their target state. Rather, the target state evolves while the objects —supposedly— are stuck in their original state.
\n![\"Convergence](\"https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kyd2lcx01jxcgq1qiq6w.png\")
Where Configuration Management tools ensure a convergence of states for mutable systems such as VMs, packaging tools such as Helm and Helmfile can be used to periodically re-synchronize containers and other immutable objects with their target state.
\nFinally, congruence is also achievable, with tools such as Argo CD. Argo CD not only deploys immutable objects to a Kubernetes cluster, but also keeps them synchronized with their last known target state, ensuring a continuous management.
\nContainers and Container Orchestrators enable fully immutable workflows for infrastructure, middleware and applications alike:
\nNodes can be managed as virtual machines, with immutable VM images deployed dynamically using IaaS;
\nMiddleware and applications can be managed as containers, with immutable OCI images deployed dynamically using Container Orchestrators.
\nAre immutable systems always the best answer? As we’ve seen, the cost in artifact management and orchestration is far from negligible. Due to the transient nature that comes with their immutability, containers are better suited for stateless applications that easily scale on clusters of neutral nodes. For this reason, highly stateful applications with long life cycles, such as databases, are still better maintained as mutable systems most of the time.
\nChoosing an immutable vs mutable architecture depends a lot on an organization’s software architecture and culture, and is not a light choice to make. Is immutable infrastructure the solution to your automation problems? Contact us, we can help you find out!
\nThis post is also available on DEV.
\n "},{"url":"/posts/how-to-manage-files-with-puppet-55e4/","relativePath":"posts/how-to-manage-files-with-puppet-55e4.md","relativeDir":"posts","base":"how-to-manage-files-with-puppet-55e4.md","name":"how-to-manage-files-with-puppet-55e4","frontmatter":{"title":"All the ways to manage files with Puppet","stackbit_url_path":"posts/how-to-manage-files-with-puppet-55e4","date":"2020-06-08T21:12:50.160Z","excerpt":"Puppet has many tools to manage configuration files. Knowing them can help you choose the one that best fits your needs.","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--X2kztkXc--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://www.camptocamp.com/wp-content/uploads/xformations_puppet1-720x400.png.pagespeed.ic.UU2oY1Zlj8.webp","comments_count":0,"positive_reactions_count":7,"tags":["puppet","cfgmgmt","tutorial","devops"],"canonical_url":"https://dev.to/camptocamp-ops/how-to-manage-files-with-puppet-55e4","template":"post"},"html":"\"Everything is a file\" is a very famous Unix principle. And because of this, most of configuration management on Unix/Linux revolves around managing files.
\nPuppet, as a configuration management tool, is no exception to this. As a consequence, there are many ways to manage configuration files with Puppet. They all have a reason to exist, and a purpose to fulfill.
\n![\"Know](\"https://dev-to-uploads.s3.amazonaws.com/i/rd2gpnxaq87rvgl5vz0j.jpg\")
Knowing your tools is the object of this blog post, with the following topics:
\nLet's start with a first choice when managing files:
\nMany practitioners I've met consider that managing whole configurations is the only acceptable way of proceeding, since managing partial configuration does not lead to a predictable final state.
\nHowever, managing whole configurations often leads to managing defaults which were carefully chosen for your GNU/Linux distribution and were perfectly fine to keep. It also leads to maintaining lots of different defaults in modules to try and stay as close as possible to the distribution standards when using default values.
\nSo both approaches have pros and cons.
\nThis is the approach you take when you want to control the full content of the software configuration. This does not mean however that everything has to fit into a single file; the configuration might be split, and splitting often makes configuration management more flexible.
\nThe easiest case is without doubt managing static content, when your file is always the same.
\nHowever simple this might seem, there can still be tricks.
\nFor example, scripts and binary blobs can easily be managed this way in Puppet, and we often see code such as:
\nfile { '/usr/local/bin/myscript.sh':\n ensure => file,\n source => "puppet:///modules/${module_name}/myscript.sh",\n}\n That works fine, but if you're trying to deploy a software (maybe even with a tarball, using the \npuppet-archive\n module), it's probably better to package it for your distribution and use your package manager (apt/yum/etc.) as the deployment layer. You'll get a much simpler Puppet code, usually better performance, and you'll rely on the package manager's metadata for idempotence:
package { 'myscript':\n ensure => present,\n}\n Another possibility is using the \npuppetlabs-vcsrepo\n resource type. The VCS (e.g. Git) will then provide the metadata to ensure idempotence, e.g.:
vcsrepo {'/usr/src/ceph-rbd-backup':\n ensure => latest,\n provider => 'git',\n source => 'https://github.com/camptocamp/ceph-rbd-backup',\n revision => 'master',\n}\n When using the\nfile\ntype to deploy static content, it is quite common to use the\nsource\nattribute to specify the file to copy:
file { '/srv/foo':\n ensure => file,\n source => "puppet:///modules/${module_name}/foo",\n}\n This makes use of the Puppetserver's fileserver feature. When using this syntax, every Puppet run will result in a\nfile_metadata\nHTTP request to the Puppetserver for each file managed, just to get the metadata necessary to decide whether the file needs to be replaced or not.
When many files are managed this way on many agents, this results in lots of HTTP requests being made during catalog application, which will saturate the Puppetserver's JRuby threads and prevent them from processing catalog compilations and reports instead.
\nInstead, when deploying non-binary content, you can use the\nfile()\nfunction with a relative path:
file { '/srv/foo':\n ensure => file,\n content => file("${module_name}/foo"),\n}\n This is totally equivalent to the previous syntax, except the whole file will be included in the catalog, instead of just a pointer to the fileserver.
\nVery often, static content is not enough to configure your software. You need variables, and a more flexible approach.
\nNative Puppet Ruby types are probably the most flexible way of managing a configuration, as they provide a very fine-grained interface to edit configuration files.
\nHowever, they do not manage the whole configuration by default. That is, unless you can use purging with them.
\nPurging resources in Puppet requires two conditions:
\nself.instances\nmethod defined)When both these conditions are met, Puppet can purge the resources it doesn't explicitly manage by:
\nself.instances\nmethod)There are two main ways of achieving this:
\nresources\ntypecrayfishx-purge\n moduleThe\nresources\ntype fits basic needs, by allowing to purge all resources not managed by Puppet. For example:
host { 'localhost':\n ensure => present,\n ip => '127.0.0.1',\n}\n\nresources { 'host':\n purge => true,\n}\n will purge all entries in\n/etc/hosts\nexcept for localhost.
The\nresources\nresource type also allows to set exceptions, though only for the\nuser\ntype:
resources { 'user':\n purge => true,\n unless_system_user => true,\n}\n This is a hard limitation, which the \npurge\ntype fixes by providing a more flexible interface, allowing to set:
For example:
\npurge { 'mount':\n state => unmounted,\n unless => ['name', '==', ['/', '/var', '/home']],\n}\n will unmount all file systems that are not managed by Puppet, unless they are mounted on\n/\n,\n/var\nor\n/home\n.
In order to manage configurations in full, the\npurge\ntype can be used with native types that manage configuration file stanzas and know how to list instances.
This is the case of:
\nhost\ntypemailalias\ntypeFor example, you can manage\nsshd_config\nin full using the \nherculesteam-augeasproviders_ssh\n module with code such as:
sshd_config {\n 'X11Forwarding':\n value => 'yes',\n ;\n\n 'UsePAM':\n value => 'no',\n ;\n}\n\npurge { 'sshd_config': }\n When there are no purgeable types for your configuration file type, and you need to manage the content from a single scope (a single Puppet class), the most obvious option is to use a simple\nfile\nresource with a template. Prefer EPP templates these days, as they are easier and safer than ERB templates:
file { '/path/to/foo':\n ensure => file,\n content => epp(\n "${module_name}/foo.epp",\n {\n var1 => 'value1',\n var2 => 'value2',\n }\n ),\n}\n When your content needs to come from multiple scopes, a single\nfile\nresource won't suffice.
If you're lucky and your configuration format supports include statements, this is the easiest way to go. For example:
\n# Deploy a static file to perform the inclusion\nfile { '/etc/sudoers':\n ensure => file,\n content => '# includedir /etc/sudoers.d',\n}\n\n# Deploy each rule as a separate file in the directory\nfile { '/etc/sudoers.d/defaults_env':\n ensure => file,\n content => 'Defaults env_reset',\n}\n\nfile { '/etc/sudoers.d/foo':\n ensure => file,\n content => 'foo ALL=(ALL:ALL) ALL',\n}\n\n# Let Puppet purge the directory of all unknown files\nfile { '/etc/sudoers.d':\n ensure => directory,\n purge => true,\n}\n Many configuration formats don't support includes: everything has to be in a single file. Managing such a file from multiple scopes requires the use of a concat module.
\nThe most used concat module is the official \npuppetlabs-concat\n. It lets you declare a target file where all fragments will be concatenated, and then deploy multiple fragments tagged for this target. For example, the sudoers example above is roughly equivalent to:
concat { '/etc/sudoers':\n ensure => present,\n}\n\nconcat::fragment { 'defaults_env':\n target => '/etc/sudoers',\n content => 'Defaults env_reset',\n order => '01',\n}\n\nconcat::fragment { 'foo':\n target => '/etc/sudoers',\n content => 'foo ALL=(ALL:ALL) ALL',\n order => '10',\n}\n Each fragment is deployed separately to the agent, then concatenated to generate the final file.
\nA few years ago, I experimented with yet another option to manage full dynamic content, without losing the benefit of sane distribution defaults.
\nThe \ncamptocamp-augeas_file\n resource type allows to use a local file on the Puppet agent as a template on which Augeas changes are applied to generate the final file:
augeas_file { '/etc/apt/sources.list.d/jessie.list':\n lens => 'Aptsources.lns',\n base => '/usr/share/doc/apt/examples/sources.list',\n changes => ['setm ./*[distribution] distribution jessie'],\n}\n Every time the Puppet agent runs, it will use\n/usr/share/doc/apt/examples/sources.list\nas a template and apply the\nchanges\nusing Augeas to generate\n/etc/apt/sources.list.d/jessie.list\n. The target file is only written if any changes occur, making it idempotent. If the template changes (e.g. after a package upgrade), the target will be regenerated.
Partial configurations have less options to be managed. They're essentially light versions of the options cited above:
\nJust as for full configurations, you can use native puppet types (\nhost\n,\nmailalias\n, Augeasproviders types, etc.), without purging them.
In addition, since you don't mind managing files partially, you can also use types which don't support purging, such as \nini_setting\n for INI file types:
ini_setting { "sample setting":\n ensure => present,\n path => '/tmp/foo.ini',\n section => 'bar',\n setting => 'baz',\n value => 'quux',\n}\n or the \nshellvar\ntype for shell configuration files:
shellvar { "ntpd options":\n ensure => present,\n target => "/etc/sysconfig/ntpd",\n variable => "OPTIONS",\n value => "-g -x -c /etc/myntp.conf",\n}\n Includes work just as for whole configurations, but without purging the directory.
\nIf no Augeasproviders exists for your resource type, but Augeas has an available lens for your configuration format, then you can most likely use the \naugeas\nresource type to manipulate it.
This is often used to manipulate XML configurations for example:
\naugeas {'foo.xml':\n incl => '/tmp/foo.xml',\n context => '/files/tmp/foo.xml/foo',\n lens => 'Xml.lns',\n changes => [\n 'set bar/# text herp',\n ],\n}\n I've kept\nfile_line\nfor the end of this list, because this is really the last option you might want to consider (just like\nexec\n) since has many downfalls.
However, if you got this far, you're probably either:
\n.bashrc\n(which the\nshellvar\ntype usually parses rather fine) or some kind of PHP or Perl configuration… I don't envy you if you have no option of using templates/concat for that!There many tools to manage files in Puppet.
\nDo you have other modules/resource types you like to use for this? Let me know in the comments!
\nThis post is also available on DEV.
\n "},{"url":"/posts/simple-secret-sharing-with-gopass-and-summon-40jk/","relativePath":"posts/simple-secret-sharing-with-gopass-and-summon-40jk.md","relativeDir":"posts","base":"simple-secret-sharing-with-gopass-and-summon-40jk.md","name":"simple-secret-sharing-with-gopass-and-summon-40jk","frontmatter":{"title":"Simple secret sharing with gopass and summon","stackbit_url_path":"posts/simple-secret-sharing-with-gopass-and-summon-40jk","date":"2020-07-28T16:34:57.621Z","excerpt":"Storing and sharing secrets doesn't have to be complex","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--t2vDCZ31--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://dev-to-uploads.s3.amazonaws.com/i/gl8147l5s2ky6po9tdfi.png","comments_count":0,"positive_reactions_count":10,"tags":["security","devops","showdev","opensource"],"canonical_url":"https://dev.to/camptocamp-ops/simple-secret-sharing-with-gopass-and-summon-40jk","template":"post"},"html":"Secrets are a fundamental, yet complex issue in software deployment.
\nSolutions such as KeepassX are simple to use, but quite impractical when it comes to automation.
\nMore complex options like Hashicorp Vault are extremely powerful, but harder to set up and maintain.
\nWhen it comes to storing securely and sharing passwords in a team, it is hard to come up with a more simple and efficient solution than Git and GnuPG combined.
\nPass is a shell script that does just that. Inside a Git repository, Pass stores passwords in individual files encrypted for all private GnuPG keys in the team. It features a CLI to manipulate passwords, add new entries, or search through existing passwords.
\nHowever, Pass is quite limited in its features, so another project was born a few years later, to provide a new Go implementation of the Pass standard. Its name: simply Gopass.
\n![\"Gopass](\"https://dev-to-uploads.s3.amazonaws.com/i/1cs4zelpc542tzc2ligm.png\")
Gopass is provided as binaries you can download from the releases page on GitHub.
\nHere are some of the features that make Gopass a great tool.
\nWhile\npass\nallows you to have a single Git repository with your passwords, Gopass lets you create multiple repositories called \"mounts\", which is very useful when you want to share different secrets with different people:
$ gopass mounts\ngopass (/home/raphink/.password-store)\n├── c2c (/home/raphink/.password-store-c2c)\n├── perso (/home/raphink/.password-store-perso)\n└── terraform (/home/raphink/.password-store-terraform)\n Gopass uses a prefix to access secrets in mounts, so\nterraform/puppet/c2c\nactually refers to the secret stored in\n/home/raphink/.password-store-terraform/puppet/c2c.gpg\n.
Each Git repository can be set to encrypt passwords for multiple GnuPG keys.
\nThe\n.gpg-id\nfile at the root of each repository contains the list of public keys to use for encryption, and the\n.public-keys/\ndirectory keeps a copy of each key, making it easy for collaborators to import them into their keyring before they can start encrypting passwords for the team.
Gopass helps you find entries when the key you gave it doesn't match an exact known path:
\n$ gopass jenkins\n[ 0] c2c/c2c_aws/jenkins-c2c\n[ 1] c2c/c2c_mgmtsrv/freeipa/c2c-jenkins-swarm\n[ 2] c2c/c2c_mgmtsrv/freeipa/jenkins-test-users\n[ 3] perso/Devel/jenkins-ci.org\n[ 4] terraform/aws/jenkins-c2c\n\nFound secrets - Please select an entry [0]: \n When decrypting a password, Gopass parses the content into two different parts: a password and a YAML document. For example, the content of a secret could look like this:
\nfoo\n---\nkey1: value1\nanother_key:\n bar: baz\n The first line of the content is the\npassword\n. If this is all you're interested in, you can use\ngopass show --password\nto retrieve it:
$ gopass show --password perso/test\nfoo\n When the second part of the content (lines 2 and following) is a valid YAML document, you can query these values by providing a key, for example:
\n$ gopass show perso/test key1\nvalue1\n Starting with Gopass 1.9.3, you can also query subkeys using either a dot or slash notation:
\n$ gopass show perso/test another_key.bar\nbaz\n$ gopass show perso/test /another_key/bar\nbaz\n This makes it extremely powerful to store several fields in the same secret.
\nGopass allows to store TOTP keys alongside passwords. For example, you can have the following secret, stored at\nterraform/service.io/api\n:
WPTmU`\n>b<Y31\n---\npassword: 'WPTmU\n`>b<Y31'\ntotp: 'PIJ6AIHETAHSHOO7SHEI1AEK6IH1SOOCHATUOSH8XUAN0OOTH9XAHRUXO4AHJAEVI'\nurl: https://myservice.io\nusername: jdoe\n In addition to retrieving each field with the corresponding key, you can also generate TOTP tokens with\ngopass totp\n:
$ gopass totp terraform/service.io/api\n568000 lasts 18s \t|------------==================|\n Gopass can be easily integrated into projects for deployments or CI/CD tasks.
\nThe easiest way to integrate Gopass is probably to use Summon.
\n![\"Summon](\"https://dev-to-uploads.s3.amazonaws.com/i/gy0cw2iqwdb85lobl5jl.png\")
Summon is a tool which dynamically exposes environment variables with values retrieved from various secret stores.\ngopass\nis one of its possible providers.
Setting it up to use\ngopass\nis rather straightforward. We use a simple wrapper called\nsummon-gopass\n, which needs to be in your PATH:
# !/bin/sh\ngopass show $(echo "${@}"|tr : \\ )\n You can also simply make\nsummon-gopass\na symbolic link to your\ngopass\nbinary, but subkeys won't work in this case.
Summon lets you provide a local\nsecrets.yml\nfile which defines which environment variables you wish to define, and how to find the values.
Here's a simple example of a\nsecrets.yml\nfile using the secret we defined earlier:
SERVICE_URL: !var terraform/service.io/api url\nUSER: !var terraform/service.io/api username\nSERVICE_PASSWORD: !var terraform/service.io/api password\n You can test this setup by running the following command in the directory containing\nsecrets.yml\n:
$ summon env\n The output should contain the 3 variables with the values stored in Gopass.
\nWhile the format above allows you to expose simple secrets as variables, it is not very practical when you need secrets exposed as files.
\nSummon covers this need however, using the\nfile\nflag. For example:
SSH_KEY: !var:file terraform/service.io/ssh private_key\n If\nterraform/service.io/ssh\nis a secret in Gopass whose\nprivate_key\nYAML field contains an SSH private key, then Summon will extract this secret, place it into a temporary file (in\n/dev/shm\nby default) and set the\nSSH_KEY\nvariable with the path to the file. After the command returns, the temporary file will be delete.
You could then use such a\nsecrets.yml\nfile with:
summon sh -c 'ssh -i $SSH_KEY user@service'\n Another useful example is to store a Kubernetes cluster configuration in Gopass, e.g.:
\n---\napiVersion: v1\nclusters:\n - cluster:\n server: https://k8s.example.com\n name: k8s\ncontexts:\n - context:\n cluster: k8s\n namespace: default\n user: default-cluster-admin\n name: default-admin\ncurrent-context: default-admin\nkind: Config\npreferences: {}\nusers:\n - name: default-cluster-admin\n user:\n token: averylongtoken\n With the following\nsecrets.yml\nfile:
KUBECONFIG: !var:file path/to/secret\n You can then work on the Kubernetes cluster with\nkubectl\nusing:
$ summon kubectl <some command>\n A simple way to pass variables to Terraform is to declare them and use\nsummon\nto pass the values:
TF_VAR_var1: !var terraform/project1/secret1 field1\n You can then run\nsummon terraform\nto dynamically pass these secrets to Terraform.
Another possibility is to use Camptocamp's Terraform Pass Provider which lets you retrieve and set passwords in Gopass natively in Terraform:
\nprovider "pass" {\n store_dir = "/srv/password-store" # defaults to $PASSWORD_STORE_DIR\n refresh_store = false # do not call `\ngit pull\n`\n}\n\n# Store a value into the Gopass store\nresource "pass_password" "test" {\n path = "secret/foo"\n password = "0123456789"\n data = {\n zip = "zap"\n }\n}\n\n# Retrieve password at another_secret/bar to be used in Terraform code\ndata "pass_password" "test" {\n path = "another_secret/bar"\n}\n The provider exposes the secret with the following properties:
\npath\n: path to the secret
password\n: secret password (first line of the content)
data\n: a structure (map) of the YAML data in the content
body\n: the content found on lines 2 and following, if it could not be parsed as YAML
full\n: the full content (all lines) of the secret
The most standard way to store secrets in Hiera is to use \nhiera-eyaml\n, which stores secret values encrypted inside YAML files, using either a PKCS7 key (default) or multiple GnuPG keys (when using \nhiera-eyaml-gpg\n).
If your passwords are already stored in Gopass, you might want to integrate that into Hiera instead.
\nThe \ncamptocamp/hiera-pass\nmodule provides two Hiera backends to retrieve keys either as full Gopass secrets, or as keys inside the secrets.
This post is also available on DEV.
\n "},{"url":"/posts/taming-puppetserver-6-a-grafana-story-3c4f/","relativePath":"posts/taming-puppetserver-6-a-grafana-story-3c4f.md","relativeDir":"posts","base":"taming-puppetserver-6-a-grafana-story-3c4f.md","name":"taming-puppetserver-6-a-grafana-story-3c4f","frontmatter":{"title":"Taming Puppetserver 6: a Grafana story","stackbit_url_path":"posts/taming-puppetserver-6-a-grafana-story-3c4f","date":"2020-05-13T08:32:41.525Z","excerpt":"Using Grafana & Catalog Diff to tune the Puppet Server","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--UmzPee8A--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://dev-to-uploads.s3.amazonaws.com/i/t2lnmj23y7z3cvgo0b1t.png","comments_count":0,"positive_reactions_count":6,"tags":["puppet","observability","containers","kubernetes"],"canonical_url":"https://dev.to/camptocamp-ops/taming-puppetserver-6-a-grafana-story-3c4f","template":"post"},"html":"After some time preparing for the migration, yesterday was finally the time to switch our first production Puppetserver to Puppet 6.
\nEverything was ready: we had been running both versions of the server alongside each other for some time, performing catalog diffs, and nothing seemed to be getting in the way as I went into ArgoCD and deployed the new version of the stack.
\n![\"Deploying](\"https://dev-to-uploads.s3.amazonaws.com/i/4yegip5ggdh0k7npdz8b.png\")
The first 30 minutes went fine. But then catalogs started failing compilation, and other services colocated on the OpenShift cluster became slow.
\nIn retrospect, I should have known something was wrong. Two weeks ago when I started my tests with Puppet 6 on another platform, I noticed that the server would die of OOM quite rapidly. Our Puppetserver 5 pods had been running happily for years with the following settings:
\nmax_active_instances: '4'\njava_xmx: '2g'\n\nrequests:\n cpu: 10m\n memory: 2Gi\nlimits:\n cpu: 3\n memory: 4Gi\n When I started new Puppet 6 instances with these settings, they would die. Initially, I thought the Xmx wasn't high enough so I set it to\n3g\nand everything seemed fine again, for the duration of the catalog-diff tests.
But when the servers started crashing in production yesterday, it was clear there was another problem. And upgrading the Xmx to a higher value didn't help.
\nSo we looked at the graphs.
\nFor years, we have been gathering internal metrics from our Puppetservers. Our Puppetserver Helm chart comes equipped with a JMX Exporter pod to gather the data and send them to Prometheus. We then have a Grafana dashboard presenting all the Puppetserver metrics in a useful manner.
\n![\"Puppetserver](\"https://dev-to-uploads.s3.amazonaws.com/i/wxq2pi5b4jn5knp4pn71.png\")
Looking at the graphs from even before the switch showed that something was indeed aloof.
\n![\"JVM](\"https://dev-to-uploads.s3.amazonaws.com/i/63f6ewg4uheojxp1abv3.png\")
Clearly, the Puppetserver 6 instances had a very different memory ramp up (blue and yellow \"teeth\" lines), even during the tests phase. I just hadn't noticed then.
\nWe started a series of tests, using Puppet Catalog Diff runs to test load the servers, and playing on all various parameters of our stack:
\nIt quickly became clear that the main factor in our problem was that the memory request was too low.
\nThe official Puppet documentation gives a rule of thumb for tuning the Puppetserver memory. It indicates that each active instance requires 512MB, but another 512MB should be provided for non-heap needs:
\n\n\nYou’ll also want to allocate a little extra heap to be used by the rest of the things going on in Puppet Server: the web server, etc. So, a good rule of thumb might be 512MB + (max-active-instances * 512MB).
\n
Our graphs clearly showed that the non-heap memory of the instances stabilized a little bit over 512MB (around 550MB as I'm writing this).
\nSince we requested 4 JRuby instances, we should ensure at least (4+1)*512MB of RAM, so 2.5GB. And while our limit was set to 4GB, the requests were only set to 2GB. Changing the requested memory to a higher value showed that this was what was making our servers misbehave.
\nWe originally set the containers to a CPU limit of 3 because or compute nodes have 4 CPUs and we wanted to leave one free.
\nWe actually noticed that Puppetserver was using closer to 2.5 CPUs as a mean. So we set the limit to 4 and saw that the Puppetserver seemed to use even less CPU, down to a mean of 2.
\nNote that limiting CPUs is necessary when running Java in containers, otherwise Java believes it runs on a single CPU.
\nThe number of JRubies recommended changed between Puppet 5 and 6, as stated in the documentation. Up to Puppet 4, it was recommended to set it to\nnum-cpus + 2\n, but the docs now state:
\n\nAs of Puppet Server 1.0.8 and 2.1.0, if you don’t provide an explicit value for this setting, we’ll default to num-cpus - 1, with a minimum value of 1 and a maximum value of 4.
\n
We ran load tests with different values of max JRuby instances and found that\nnum-cpus - 1\nwas indeed the best good value.
Most importantly, we found that setting up the max JRuby instances higher than the number of CPUs made compilation time quite slower (supposedly as it would increase context-switch between the instances).
\nFollowing the guidelines and our tests, we ended up with:
\nmax_active_instances: '3' # since we have 4 CPUs\njava_xmx: '2g' # (3+1)*512MB\n\nrequests:\n cpu: 10m\n memory: 3Gi # 1.5*XmX\nlimits:\n cpu: 4 # Use all available CPUs as limit\n memory: 3.3Gi # 1.1*request just in case\n On this graph from the last 2 days, we can clearly see the situations before production time, during crisis, and after proper tuning:
\n![\"JVM](\"https://dev-to-uploads.s3.amazonaws.com/i/t2lnmj23y7z3cvgo0b1t.png\")
So our Puppetserver was back under control, with pretty similar memory settings to what Puppet 5 used.
\nKeeping the two pods we had been running, with their new tuned settings, we ran stress tests by setting the number of parallel threads in the Catalog Diff run.
\nWhen running 12 threads in parallel (far more than what 2 pods with 3 JRubies each can take), we noticed something I had seen before but not understood:
\nFailed to retrieve catalog for foo.example.com from puppetserver in environment catalog_diff: Failed to open TCP connection to puppetserver:8140 (No route to host - connect(2) for "puppetserver" port 8140)\n I had initially thought this was what happened by the Puppetserver was too busy and started rejecting connections. But no, this was clearly another problem, linked to Kubernetes networking and readiness probes.
\nAs we kept an eye on the Pods readiness during the run, we noticed the pods were going on and off, and thus being taken out of the Kubernetes service on a regular basis. The TCP connection issues happened when both pods were taken out of the service at the same time, since the service ended up with no endpoints left.
\nSo we turned to the pod's readiness probe to tune it.
\nThis is the default readiness probe we use in our Helm chart:
\nreadinessProbe:\n httpGet:\n path: /production/status/test\n port: http\n scheme: HTTPS\n httpHeaders:\n - name: Accept\n value: pson\n initialDelaySeconds: 30\n The initial delay lets the Puppetserver start its first JRuby instance before sending a probe. The default in Kubernetes would be\n0\notherwise, which would clearly fail for a Puppetserver.
For all other settings, we relied on the defaults. As per Kubernetes docs:
\n\n\n\n
\n- initialDelaySeconds: Number of seconds after the container has started before liveness or readiness probes are initiated. Defaults to 0 seconds. Minimum value is 0.
\n- periodSeconds: How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.
\n- timeoutSeconds: Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1.
\n- successThreshold: Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1.
\n- failureThreshold: When a Pod starts and the probe fails, Kubernetes will try failureThreshold times before giving up. Giving up in case of liveness probe means restarting the container. In case of readiness probe the Pod will be marked Unready. Defaults to 3. Minimum value is 1.
\n
The\ntimeoutSeconds\nparameter looks very low at 1 second by default. Indeed, we know that a busy Puppetserver could take more than 1 second to respond and that would be perfectly acceptable. So we've set it to\n5\ninstead and the service has been much more stable since.
We've also set\nfailureThreshold\nto\n5\n.
A small fine-tuning goes a long way!
\nBe sure to gather enough data about your Puppetserver so you have the tools to debug its behavior when you need.
\nDo you have Puppet, Kubernetes, or observability needs? You can contact us and we'll be happy to put our expertise at your service!
\nThis post is also available on DEV.
\n "},{"url":"/posts/templating-puppet-control-repositories-3pk7/","relativePath":"posts/templating-puppet-control-repositories-3pk7.md","relativeDir":"posts","base":"templating-puppet-control-repositories-3pk7.md","name":"templating-puppet-control-repositories-3pk7","frontmatter":{"title":"Templating Puppet Control Repositories","stackbit_url_path":"posts/templating-puppet-control-repositories-3pk7","date":"2020-07-21T08:45:28.730Z","excerpt":"When managing multiple Puppet Control Repositories, modulesync is a very useful tool to keep files in sync.","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--X2kztkXc--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://www.camptocamp.com/wp-content/uploads/xformations_puppet1-720x400.png.pagespeed.ic.UU2oY1Zlj8.webp","comments_count":3,"positive_reactions_count":1,"tags":["puppet","devops","cfgmgmt","tutorial"],"canonical_url":"https://dev.to/camptocamp-ops/templating-puppet-control-repositories-3pk7","template":"post"},"html":"Puppet code is usually deployed using a Control Repository, a single Git repository used by R10k (or Code Manager on Puppet Enterprise) to manage Puppet environments on Puppet Masters.
\nOn complex infrastructures with multiple independent Puppet Masters, you might have the need to use multiple control repositories. For example at Camptocamp, we have specific clients with enough nodes to have their own Puppet infrastructure each.
\nFor these clients, we do not want to use a shared Puppet Control Repository. However, we do want to keep the code as similar as possible between the infrastructures, and make sure some parameters and settings (admin accounts, ssh keys, etc.) are synchronized.
\nModulesync is a piece of software initially created by Puppet Inc. to synchronize files between Git repositories for Puppet modules. Nowadays, this feature is being served by PDK for Puppet modules, so modulesync is now managed by the Vox Pupuli community.
\nFor years, we have been using it at Camptocamp to keep our Control Repositories synchronized.
\nIn order to achieve this, we use a template repository, which we call\npuppetmaster-common\n.
Each of our clients has their own GitLab instance with their Puppet Control Repository, and this template repository brings it all together.
\nThis repository is set as follows.
\nThis file contains the general settings for modulesync:
\n---\n# default namespace in GitLab instances\nnamespace: 'camptocamp'\n# Branch to synchronize\nbranch: 'msync'\n# Default Merge Request title\npr_title: 'Modulesync [autodiff]'\n# Default Merge Request target branch\npr_target_branch: 'staging'\n On all our Control Repositories, we have locked the\nstable\nand\nstaging\nbranches to prevent pushes to them. This forces us to create Merge Requests for new features, ensuring quality through our CI pipeline.
For this reason, we use a separate branch, called\nmsync\n, to perform the synchronizations.
Since we use several GitLab instances and we want to be able to automate Merge Request creation, this file contains GitLab API URLs and tokens per managed Control Repository. It looks similar to this:
\npuppetmaster-c2c:\n :remote: 'ssh://git@gitlab1/camptocamp/is/puppet/puppetmaster-c2c.git'\n :namespace: 'camptocamp/is/puppet'\n :gitlab:\n :token: 'abc123def456'\n :base_url: 'https://gitlab1/api/v4'\n\npuppetmaster-client1:\n :remote: 'ssh://git@gitlab-client1/puppet/puppetmaster-client1.git'\n :namespace: 'puppet'\n :gitlab:\n :token: 'someOtherToken'\n :base_url: 'https://gitlab-client1/api/v4'\n The\nmoduleroot\ndirectory contains the files we want to synchronize, as ERB templates. In our case:
moduleroot/\n├── doc\n│ ├── architecture.md.erb\n│ └── before_after.md.erb\n├── environment.conf.erb\n├── Gemfile.erb\n├── .gitignore.erb\n├── .gitlab-ci.yml.erb\n├── hieradata\n│ └── cross-site\n│ ├── common-cross-site.yaml.erb\n│ ├── README.md.erb\n│ ├── .travis.yml.erb\n│ └── verify-key-length.erb\n├── hiera-eyaml-gpg.recipients.erb\n├── Puppetfile.erb\n├── .puppet-lint.rc.erb\n├── Rakefile.erb\n├── README.md.erb\n└── scripts\n ├── bolt.erb\n ├── docker.erb\n ├── node_deactivate.erb\n ├── puppetca.erb\n └── puppet-query.erb\n A few notes here on the files here.
\nMost of these files (e.g. the scripts,\nGemfile\n, or\nenvironment.conf\n) are actually static, but they need to be named\n.erb\nnonetheless, otherwise\nmodulesync\nwill ignore them.
hiera-eyaml-gpg.recipients.erb\nworks essentially as a filter on the\nhiera-eyaml-gpg.recipients\nfile at the top of the repository, taking every admin key, as well as one\npuppet@\nkey specified in the\n.sync.yml\nof the control repository with the\nmaster_gpg_key\nsetting:
<%=\nbasedir = File.expand_path('..', File.dirname(__FILE__))\nrecipients_file = File.expand_path(File.join(basedir, 'hiera-eyaml-gpg.recipients'))\n\nFile.readlines(recipients_file).map { |l|\n r = l.strip\n if r =~ /^puppet@/\n r if @configs['master_gpg_key'] == r\n else\n r\n end\n}.compact.join("\\n")\n%>\n Similar to\nhiera-eyaml-gpg.recipients\n,\nPuppetfile\nis managed as a filter. We keep a full\nPuppetfile\nat the top of the repository, with all the modules we use on all Puppet Infrastructures, and the default versions we want. Then each Control Repository can pick which module to include and optionally override versions.
The\nPuppetfile.erb\ntemplate uses Augeas to cleanly filter and rewrite the target\nPuppetfile\n:
############################################### \n# This file is managed in puppetmaster-common # \n# Do not edit locally # \n############################################### \n\n<%= require 'augeas'\nbasedir = File.expand_path('..', File.dirname(__FILE__))\nbase_pf = File.expand_path(File.join(basedir, 'Puppetfile'))\nbase_pf_content = File.read(base_pf)\nlens_dir = File.expand_path(File.join(basedir, 'lenses'))\n\ndef mod_regexp(name)\n "*[label()!='# comment' and .=~regexp('([^/-]+[/-])?# {name}')]"\nend\n\nAugeas.open(nil, lens_dir, Augeas::NO_MODL_AUTOLOAD) do |aug|\n aug.set('/input', base_pf_content)\n unless aug.text_store('Puppetfile.lns', '/input', '/parsed')\n msg = aug.get('/augeas//error')\n fail "Failed to parse common Puppetfile: # {msg}"\n end\n aug.set('/augeas/context', '/parsed')\n all_modules = aug.match('*[label()!="# comment"]').map { |m| aug.get(m).split(%r{[/-]}).last }\n\n whitelist = @configs['modules'].keys if @configs['modules']\n not_in_all = whitelist - all_modules if whitelist\n fail "Module(s) # {not_in_all.join(', ')} not found in common Puppetfile" if not_in_all and !not_in_all.empty?\n\n # Remove unnecessary modules\n (all_modules - whitelist).each do |m|\n aug.rm(mod_regexp(m))\n end if whitelist\n\n # Amend\n modified = @configs['modules'].reject { |m, v| v.nil? } if @configs['modules']\n modified.each do |m, c|\n aug.set(mod_regexp(m), "# {c['user']}/# {m}") if c['user']\n if c['version']\n aug.rm("# {mod_regexp(m)}/git")\n aug.rm("# {mod_regexp(m)}/ref")\n aug.set("# {mod_regexp(m)}/@version", c['version'])\n else\n aug.rm("# {mod_regexp(m)}/@version")\n aug.set("# {mod_regexp(m)}/git", c['git']) if c['git']\n aug.set("# {mod_regexp(m)}/ref", c['ref']) if c['ref']\n end\n end if modified\n\n aug.text_retrieve('Puppetfile.lns', '/input', '/parsed', '/output')\n unless aug.match('/augeas/text/parsed/error').empty?\n fail "Failed to generate Puppetfile: # {aug.get('/augeas/text/parsed/error')}\n # {aug.get('/augeas/text/parsed/error/message')}"\n end\n aug.get('/output')\nend -%>\n This file defines the CI/CD pipelines for our Control Repositories, extending our generic Puppet pipelines rules. It takes variables to control catalog-diff.
\nThe cross-site hieradata level contains common system accounts with their UID, shell & SSH key. We then use our accounts module to deploy these accounts.
\nEach Control Repository features a\n.sync.yml\nfile to provide overrides for variables. Here's an example:
---\nRakefile:\n master_gpg_key: 'puppet@client1'\n.gitlab-ci.yml:\n puppetdb_urls: 'https://puppetdb.client1.ch'\n puppet_server: 'puppet.client1.ch'\n puppetdiff_url: 'https://puppetdiff.client1.ch'\nPuppetfile:\n modules:\n # include accounts module, with default version\n accounts:\n # include letsencrypt module, override version\n letsencrypt:\n git: 'https://github.com/saimonn/puppet-letsencrypt'\n ref: 'default_cert_name'\n Since\nmanaged_modules.yml\ncontains secret tokens for the various GitLabs, we don't want to commit it to the Git repository. Instead, the content of this file is stored in \ngopass\n and retrieved dynamically with \nsummon\n.
In order to use\nsummon\n, we have a local\nsecrets.yml\npointing to the location of the\nmanaged_modules.yml\nfile in\ngopass\n:
---\nMSYNC_MANAGED_MODULES: !var:file puppet/msync/managed_modules\n and use a\nmsync_update\nwrapper to launch\nmodulesync\n:
# !/bin/bash\n\nbundle exec msync update --managed_modules_conf=$MSYNC_MANAGED_MODULES "$@"\n This then allows to test the changes with:
\n$ summon ./msync_update -m "Update module foo" --noop\n and then deploy on a single site (or all without the filter):
\n$ summon ./msync_update -m "Update module foo" -f c2c --pr\n Do you have specific Puppet needs? Contact us, we can help you!
\nThis post is also available on DEV.
\n "},{"url":"/posts/visibility-comments-b65/","relativePath":"posts/visibility-comments-b65.md","relativeDir":"posts","base":"visibility-comments-b65.md","name":"visibility-comments-b65","frontmatter":{"title":"How to encourage interaction on dev.to posts?","stackbit_url_path":"posts/visibility-comments-b65","date":"2020-06-10T19:19:35.493Z","excerpt":"After a few years of being inactive on dev.to, I've started actively posting about a month ago. I se...","thumb_img_path":"https://res.cloudinary.com/practicaldev/image/fetch/s--BobVuS2v--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://dev-to-uploads.s3.amazonaws.com/i/w14l9yddz1nqrnvo3xfe.jpg","comments_count":0,"positive_reactions_count":1,"tags":["discuss","writing","beginners","question"],"canonical_url":"https://dev.to/raphink/visibility-comments-b65","template":"post"},"html":"After a few years of being inactive on dev.to, I've started actively posting about a month ago.
\nI see quite a few visits to some of my posts (up to nearly 400 views for some), but I get absolutely no comments (unless it's a small post with the\n# discuss\ntag).
Is that normal? Is there a way on dev.to to drive more reactions/interactions? Is it the subject (mainly DevOps/SRE-related) that is not core to this platform?
\nAm I missing something?
\n\n "}],"site":{"siteMetadata":{"description":"A minimal blogging theme for Stackbit","header":{"title":"Raphaël Pinson","tagline":"Infrastructure Developer working @camptocamp. ","background_img":"images/header-bg.jpg","has_nav":true,"nav_links":[{"label":"Home","url":"https://raphink.info","type":"link"},{"label":"Contact","url":"/contact/","type":"link"}],"has_social":true,"social_links":[{"label":"GitHub","url":"https://github.com/raphink","type":"icon","icon_class":"fa-github","new_window":true},{"label":"DEV","url":"https://dev.to/raphink","type":"icon","icon_class":"fa-dev","new_window":true},{"label":"Twitter","url":"https://twitter.com/raphink","type":"icon","icon_class":"fa-twitter","new_window":true},{"label":"LinkedIn","url":"https://www.linkedin.com/in/raphink","type":"icon","icon_class":"fa-linkedin","new_window":true},{"label":"Stack Exchange","url":"https://stackexchange.com/users/82664/%e2%84%9daphink","type":"icon","icon_class":"fa-stack-exchange","new_window":true}]},"footer":{"content":"© All rights reserved.","links":[{"label":"Made with Stackbit.","url":"https://www.stackbit.com","type":"link","new_window":true},{"label":"Generated from DEV","url":"https://dev.to/connecting-with-stackbit","new_window":true,"type":"link"}]},"palette":"yellow","title":"Open Source Automation"},"pathPrefix":"","data":{"data":{"author":{"name":"Raphaël Pinson","avatar":"https://res.cloudinary.com/practicaldev/image/fetch/s--BG4LRVnz--/c_fill,f_auto,fl_progressive,h_640,q_auto,w_640/https://dev-to-uploads.s3.amazonaws.com/uploads/user/profile_image/59811/cdcdbc95-1306-4455-9f79-fa032c300206.jpeg"},"social":{"devto":{"username":"raphink"},"github":{"username":"raphink"}}}}},"menus":{}}}