Scripting

13 Posts
2 Sections
2 Types

October 31, 2024 7 min read Posts

👷👮 Makefile VS Justfile

Posts Devops Command-Liner Scripting

The two task-runners comparison!

August 1, 2023 1 min read Docs

🌅 UV

Docs Devops Scripting Python Repository

Install 1# curl method 2curl -LsSf https://astral.sh/uv/install.sh | sh 3 4# Pip method 5pip install uv Quick example 1pyenv install 3.12 2pyenv local 3.12 3python -m venv .venv 4source .venv/bin/activate 5pip install pandas 6python 7 8# equivalent in uv 9uv run --python 3.12 --with pandas python Usefull 1uv python list --only-installed 2uv python install 3.12 3uv venv /path/to/environment --python 3.12 4uv pip install django 5uv pip compile requirements.in -o requirements.txt 6 7uv init myproject 8uv sync 9uv run manage.py runserver Run as script Put before the import statements: 1#!/usr/bin/env -S uv run --script 2# /// script 3# requires-python = ">=3.12" 4# dependencies = [ 5# "ffmpeg-normalize", 6# ] 7# /// Then can be run with uv run sync-flickr-dates.py. uv will create a Python 3.12 venv for us. For me this is in ~/.cache/uv (which you can find via uv cache dir).

Read more about 🌅 UV.

August 1, 2023 1 min read Docs

🐍 Cobra

Docs Devops Scripting Golang Cobra

A Command Builder for Go Usefull Installation 1# Install GO 2GO_VERSION="1.21.0" 3wget https://go.dev/dl/go${GO_VERSION}.linux-amd64.tar.gz 4sudo tar -C /usr/local -xzf go${GO_VERSION}.linux-amd64.tar.gz 5export PATH=$PATH:/usr/local/go/bin 6 7# Install Cobra - CLI builder 8go install github.com/spf13/cobra-cli@latest 9sudo cp -pr ./go /usr/local/. Init 1mkdir -p ${project} && cd ${project} 2go mod init ${project} 3cobra-cli init 4go build 5go install 6cobra-cli add timezone

August 1, 2023 3 min read Docs

🐣 Bash Functions for k8s

Docs Devops Shell Scripting Kubernetes

A list of nice findings for Kubernetes List all images in Helm chart 1images=$(helm template -g $helm |yq -N '..|.image? | select(. == "*" and . != null)'|sort|uniq|grep ":"|egrep -v '*:[[:blank:]]' || echo "") upload images listed in an Helm chart 1load_helm_images(){ 2 # look in helm charts 3 for helm in $(ls ../../roles/*/files/helm/*.tgz); do 4 printf "\e[1;34m[INFO]\e[m Look for images in ${helm}...\n" 5 6 images=$(helm template -g $helm |yq -N '..|.image? | select(. == "*" and . != null)'|sort|uniq|grep ":"|egrep -v '*:[[:blank:]]' || echo "") 7 8 dir=$( dirname $helm | xargs dirname ) 9 10 echo "####" 11 12 if [ "$images" != "" ]; then 13 printf "\e[1;34m[INFO]\e[m Images found in the helm charts: ${images}\n" 14 printf "\e[1;34m[INFO]\e[m Create ${dir}/images images...\n" 15 16 mkdir -p ${dir}/images 17 18 while i= read -r image_name; do 19 archive_name=$(basename -a $(awk -F : '{print $1}'<<<${image_name})); 20 printf "\e[1;34m[INFO]\e[m Pull images...\n" 21 podman pull ${image_name}; 22 printf "\e[1;34m[INFO]\e[m Push ${image_name} in ${dir}/images/${archive_name}\n" 23 podman save ${image_name} --format oci-archive -o ${dir}/images/${archive_name}; 24 done <<< ${images} 25 else 26 printf "\e[1;34m[INFO]\e[m No Images found in the helm charts: $helm\n" 27 fi 28 done 29} Check components version 1function checkComponentsInstall() { 2 componentsArray=("kubectl" "helm") 3 for i in "${componentsArray[@]}"; do 4 command -v "${i}" >/dev/null 2>&1 || 5 { echo "[ERROR] ${i} is required, but it's not installed. Aborting." >&2; exit 1; } 6 done 7} Version comparator 1function checkK8sVersion() { 2 currentK8sVersion=$(kubectl version --short | grep "Server Version" | awk '{gsub(/v/,$5)}1 {print $3}') 3 testVersionComparator 1.20 "$currentK8sVersion" '<' 4 if [[ $k8sVersion == "ok" ]]; then 5 echo "current kubernetes version is ok" 6 else 7 minikube start --kubernetes-version=v1.22.4; 8 fi 9} 10 11 12# the comparator based on https://stackoverflow.com/a/4025065 13versionComparator () { 14 if [[ $1 == $2 ]] 15 then 16 return 0 17 fi 18 local IFS=. 19 local i ver1=($1) ver2=($2) 20 # fill empty fields in ver1 with zeros 21 for ((i=${#ver1[@]}; i<${#ver2[@]}; i++)) 22 do 23 ver1[i]=0 24 done 25 for ((i=0; i<${#ver1[@]}; i++)) 26 do 27 if [[ -z ${ver2[i]} ]] 28 then 29 # fill empty fields in ver2 with zeros 30 ver2[i]=0 31 fi 32 if ((10#${ver1[i]} > 10#${ver2[i]})) 33 then 34 return 1 35 fi 36 if ((10#${ver1[i]} < 10#${ver2[i]})) 37 then 38 return 2 39 fi 40 done 41 return 0 42} 43 44testVersionComparator () { 45 versionComparator $1 $2 46 case $? in 47 0) op='=';; 48 1) op='>';; 49 2) op='<';; 50 esac 51 if [[ $op != "$3" ]] 52 then 53 echo "Kubernetes test fail: Expected '$3', Actual '$op', Arg1 '$1', Arg2 '$2'" 54 k8sVersion="not ok" 55 else 56 echo "Kubernetes test pass: '$1 $op $2'" 57 k8sVersion="ok" 58 fi 59}

August 1, 2023 3 min read Docs

🐦 Awk

Docs SysAdmin Scripting Shell

The Basics awk is treat each line as a table, by default space are separators of columns. General syntax is awk 'search {action}' file_to_parse. 1# Give the value higher than 75000 in column $4 2df | awk '$4 > 75000' 3 4# Print the all line when column $4 is higher than 75000 5df | awk '$4 > 75000 {print $0}' But if you look for a string, the search need to be included in /search/ or ;search;. When you print $0 represent the all line, $1 first column, $2 second column etc.

Read more about 🐦 Awk.

August 1, 2023 1 min read Docs

🐴 Sed

Docs SysAdmin Scripting Shell

The Basics 1sed -e '…' -e '…' # Several execution 2sed -i # Replace in place 3sed -r # Play with REGEX 4 5# The most usefull 6sed -e '/^[ ]*#/d' -e '/^$/d' <fich.> # openfile without empty or commented lines 7sed 's/ -/\n -/g' # replace all "-" with new lines 8sed 's/my_match.*/ /g' # remove from the match till end of line 9sed -i '4048d;3375d' ~/.ssh/known_hosts # delete lines Number 10 11# Buffer 12s/.*@(.*)/$1/; # keep what is after @ put it in buffer ( ) and reuse it with $1. 13sed -e '/^;/! s/.*-reserv.*/; Reserved: &/' file.txt # resuse search with & 14 15# Search a line 16sed -e '/192.168.130/ s/^/#/g' -i /etc/hosts # Comment a line 17sed -re 's/^;(r|R)eserved:/; Reserved:/g' file.txt # Search several string 18 19# Insert - add two lines below a match pattern 20sed -i '/.*\"description\".*/s/$/ \n \"after\" : \"network.target\"\,\n \"requires\" : \"network.target\"\,/g' my_File 21 22# Append 23sed '/WORD/ a Add this line after every line with WORD' 24 25# if no occurence, then add it after "use_authtok" 26sed -e '/remember=10/!s/use_authtok/& remember=10/' -i /etc/pam.d/system-auth-permanent

August 1, 2023 3 min read Docs

👮 CUE-lang

Docs Devops Scripting Golang Data

CUE stands for Configure, Unify, Execute Basics Installation 1# Install GO 2GO_VERSION="1.21.0" 3wget https://go.dev/dl/go${GO_VERSION}.linux-amd64.tar.gz 4sudo tar -C /usr/local -xzf go${GO_VERSION}.linux-amd64.tar.gz 5export PATH=$PATH:/usr/local/go/bin 6 7go install cuelang.org/go/cmd/cue@latest 8sudo cp -pr ./go /usr/local/. 9 10# or use Container 11printf "\e[1;34m[INFO]\e[m Install CUElang:\n"; 12podman pull docker.io/cuelang/cue:latest concepts top -> schema -> constraint -> data -> bottom Command 1# import a file 2cue import imageset-config.yaml 3 4# Validate 5cue vet imageset-config.cue imageset-config.yaml 6 7 8* Some basics example 9 10```go 11// This is a comment 12_greeting: "Welcome" // Hidden fields start with "_" 13#project: "CUE" // Definitions start with "#" 14 15message: "\(_greeting) to \(#project)!" // Regular fields are exported 16 17#Person: { 18 age: number // Mandatory condition and must be a number 19 hobbies?: [...string] // non mandatory but if present must be a list of string 20} 21 22// Constrain which call #Person and check if age 23#Adult: #Person & { 24 age: >=18 25} 26 27// =~ match a regular expression 28#Phone: string & =~ "[0-9]+" 29 30// Mapping 31instanceType: { 32 web: "small" 33 app: "medium" 34 db: "large" 35} 36 37server1: { 38 role: "app" 39 instance: instanceType[role] 40} 41 42// server1.instance: "medium" Scripting 1# executable have extension name "_tool.cue" 2 3# usage 4cue cmd prompter 1package foo 2 3import ( 4 "tool/cli" 5 "tool/exec" 6 "tool/file" 7) 8 9// moved to the data.cue file to show how we can reference "pure" Cue files 10city: "Amsterdam" 11 12// A command named "prompter" 13command: prompter: { 14 15 // save transcript to this file 16 var: { 17 file: *"out.txt" | string @tag(file) 18 } // you can use "-t flag=filename.txt" to change the output file, see "cue help injection" for more details 19 20 // prompt the user for some input 21 ask: cli.Ask & { 22 prompt: "What is your name?" 23 response: string 24 } 25 26 // run an external command, starts after ask 27 echo: exec.Run & { 28 // note the reference to ask and city here 29 cmd: ["echo", "Hello", ask.response + "!", "Have you been to", city + "?"] 30 stdout: string // capture stdout, don't print to the terminal 31 } 32 33 // append to a file, starts after echo 34 append: file.Append & { 35 filename: var.file 36 contents: echo.stdout // because we reference the echo task 37 } 38 39 // also starts after echo, and concurrently with append 40 print: cli.Print & { 41 text: echo.stdout // write the output to the terminal since we captured it previously 42 } 43} Sources Offical Documentation