๐Ÿ“Š Process Monitoring (pidstat)
๐Ÿ“Š Process Monitoring (pidstat)
pidstat reports the CPU, memory, I/O and context-switch activity of processes. Report the process context-switching activity 1# pidstat -w -p 3446 2 5 2Linux 3.10.0-123.13.2.el7.x86_64 (localhost.localdomain) 12/26/2014 3_x86_64_ (1 CPU) 407:23:38 AM UID PID cswch/s nvcswch/s Command 507:23:40 AM 0 3446 0.50 0.00 sshd 607:23:42 AM 0 3446 0.50 0.00 sshd 707:23:44 AM 0 3446 0.50 0.00 sshd 807:23:46 AM 0 3446 0.50 0.00 sshd 907:23:48 AM 0 3446 0.50 0.00 sshd 10Average: 0 3446 0.50 0.00 sshd cswch/s : number of voluntary context switches the task made per second. (A voluntary context switch occurs when a task blocks because it requires a resource that is unavailable.) nvcswch/s : number of non-voluntary context switches the task made per second. (An involuntary context switch takes place when a task executes for the duration of its time slice and is then forced to relinquish the processor.) Page faults and memory 1pidstat -r -p <PID> 3600 72 # every hour, 72 times - practical for long-term monitoring. 2 3pidstat -r -p <PID> 50 12 407:26:44 PM PID minflt/s majflt/s VSZ RSS %MEM Command 507:27:34 PM 13775 1.64 0.00 34957320 18183312 55.30 java minflt/s : number of minor faults the task has made per second โ€” those which did not require loading a memory page from disk. majflt/s : number of major faults the task has made per second โ€” those which required loading a memory page from disk. VSZ : Virtual Size โ€” the virtual memory usage of the entire task in kilobytes. RSS : Resident Set Size โ€” the non-swapped physical memory used by the task in kilobytes. Disk I/O 1pidstat -d -p <PID> 50 12 pidstat -d reports I/O statistics (kernels 2.6.20 and later only). The following values are displayed:
๐Ÿ” Find & Inspect Processes
๐Ÿ” Find & Inspect Processes
Find a process 1ps -fp <pid> # find a process by its PID. 2pidof httpd # find the PIDs of httpd. 3pidstat -lp <pid> # process name with all its complete arguments. 4 # for a tomcat or weblogic process, you can split the arguments with: sed 's/ -/\n -/g' 5pidstat -C "mysql" # find a process by its name (gives the PID and CPU load). The process tree 1pstree -pu # the process tree with PID and user (if pstree is not installed, use the alternatives below). 1ps -ejH 2 PID PGID SID TTY TIME CMD 3 1 1 1 ? 00:00:37 init 411016 11009 11009 ? 00:00:00 sshd 511017 11017 11017 pts/12 00:00:00 bash 611125 11125 11017 pts/12 00:00:00 telnet 1ps axjf 2 PPID PID PGID SID TTY TPGID STAT UID TIME COMMAND 3 0 1 1 1 ? -1 Ss 0 0:37 init [5] 4 8617 10610 10610 10610 ? -1 Ss 0 0:00 \_ sshd: support [priv] 510610 10710 10610 10610 ? -1 S 5027 0:00 \_ sshd: support@notty 610710 10711 10711 10711 ? -1 Ss 5027 0:00 \_ sshd: support@internal-sftp-server 1ps faux 2USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND 3root 1 0.0 0.0 10372 696 ? Ss Aug09 0:37 init [5] 4user1 4168 0.0 0.0 8728 968 ? Ss Aug24 0:00 | \_ /bin/bash -c perl /data/supports/scripts/SRAM_asr5k.pl &>/dev/null 5user1 4174 0.0 0.0 34068 5044 ? S Aug24 0:00 | \_ perl /data/supports/SRAM_asr5k.pl 6user1 4188 0.0 0.0 8728 984 ? S Aug24 0:00 | \_ sh -c grep -c SRAM /data/syslogCOLLECT/LTE_*/*/*.20160824.log /proc The /proc filesystem exposes per-process information:
๐Ÿ–ฅ Out-of-Band Management
๐Ÿ–ฅ Out-of-Band Management
Out-of-band management refers to accessing and controlling a server (console, power on/off, BIOS, remote media) through a dedicated management channel that is separate from the normal data path. “In-band” means going through the OS and its network interface; “out-of-band” uses an independent controller (a BMC โ€” Baseboard Management Controller) with its own network port, so it still works even when the OS is down, the machine is hung, or the network stack is broken. RSA is one vendor/technology family of out-of-band access.
๐Ÿ” ISO Checksum
๐Ÿ” ISO Checksum
Verify an ISO image To verify that an ISO image is good: compute its checksum with a sha1 or sha256 key, then compare it with the key published on the official website. 1sha256sum image.iso 2sha1sum image.iso
๐Ÿš€ KickStart
๐Ÿš€ KickStart
KickStart Technology that allows deploying servers with a predefined configuration (Red Hat’s equivalent of Solaris JumpStart). Default volume manager : LVM. See the LVM page for details.
๐Ÿง Unix Families
๐Ÿง Unix Families
The Unix variants Unix proprietary GNU / Linux BSD / open source Mainframe Virtualisation AIX Debian FreeBSD MVS (IBM) VMware / VirtualBox HP-UX Slackware NetBSD SCOS (Bull) Cloud (IaaS) SunOS (BSD fork) โ†’ Solaris SUSE OpenBSD OpenStack (IaaS/SaaS) IRIX (SGI) Red Hat FreeBSD โ†’ macOS Fedora openSUSE CentOS Ubuntu (Debian) Mint The Unix families Early SunOS (1โ€“4) was BSD-derived; from Solaris 2 / SunOS 5 onward it is System V (SVR4) based โ€” hence Solaris is listed under System V while the variants table notes its BSD fork. Historically, Unix split into two main branches:
๐Ÿ“ˆ Performance Monitoring & Tuning
๐Ÿ“ˆ Performance Monitoring & Tuning
How to approach performance monitoring and tuning in Linux, and the various subsystems (and performance metrics) that need to be monitored. On a very high level, the following four subsystems need to be monitored: CPU Memory I/O Network 1. CPU Four critical performance metrics for the CPU: context switch, run queue, CPU utilization, and load average. Context Switch When the CPU switches from one process (or thread) to another, it is called a context switch. When a process switch happens, the kernel stores the current state of the CPU (of a process or thread) in memory. The kernel also retrieves the previously stored state (of a process or thread) from memory and puts it in the CPU. Context switching is essential for multitasking of the CPU. However, a higher level of context switching can cause performance issues. Run Queue The run queue indicates the total number of active processes in the current queue for the CPU. When the CPU is ready to execute a process, it picks it up from the run queue based on the priority of the process. Note that processes that are in a sleep state, or I/O wait state, are not in the run queue. A higher number of processes in the run queue can therefore cause performance issues. CPU Utilization Indicates how much of the CPU is currently being used. 100% CPU utilization means the system is fully loaded. Load Average Indicates the average CPU load over a specific time period. On Linux, load average is displayed for the last 1 minute, 5 minutes, and 15 minutes. This helps to see whether the overall load on the system is going up or down. For example, a load average of 0.75 1.70 2.10 indicates that the load is coming down (0.75 = last 1 minute, 1.70 = last 5 minutes, 2.10 = last 15 minutes). Note that this load average is calculated by combining both the total number of processes in the queue, and the total number of processes in the uninterruptible task status. 2. Network A good understanding of TCP/IP concepts is helpful when analyzing any network issue. For network interfaces, monitor the total number of packets (and bytes) received/sent through the interface, the number of packets dropped, etc. 3. I/O I/O wait is the amount of time the CPU is waiting for I/O. Consistent high I/O wait on the system indicates a problem in the disk subsystem. Monitor reads/second and writes/second. These are measured in blocks, i.e. the number of blocks read/written per second. They are also referred to as bi and bo (block in and block out). tps indicates total transactions per second, which is the sum of rtps (read transactions per second) and wtps (write transactions per second). 4. Memory RAM is the physical memory. If you have 4 GB of RAM installed, you have 4 GB of physical memory. Virtual memory = swap space available on disk + physical memory. The virtual memory contains both user space and kernel space. Using a 32-bit or a 64-bit system makes a big difference in determining how much memory a process can use: On a 32-bit system a process can only access a maximum of 4 GB of virtual memory. On a 64-bit system there is no such limitation. Unused RAM is used by the kernel as filesystem cache. Linux swaps when it needs more memory than the physical memory. When it swaps, it writes the least-used memory pages from the physical memory to the swap space on the disk. Lots of swapping can cause performance issues: the disk is much slower than the physical memory, and it takes time to swap the memory pages from RAM to disk. The subsystems are interrelated All four subsystems are interrelated. Just because you see a high reads/second, writes/second, or I/O wait, it does not mean the issue is with the I/O subsystem. It also depends on what the application is doing. In most cases, the performance issue is caused by the application running on the Linux system.
๐Ÿงน Disk Cleanup
๐Ÿงน Disk Cleanup
Find old files 1find . -type f -mtime +150 -exec ls -lrt {} \; | more 2find . -maxdepth 1 -name "*.log" -mtime +10 -exec ls -lrt {} \; 3find . -mtime +150 -exec rm -f {} \; The find loop is cheaper than a shell loop (for ...). You can make it even more efficient by batching the rm calls: 1find . -type f -print -exec rm -- "{}" + # note the "+" instead of the usual "\;" See which directories use the most space 1du -max . # list all the FS sub-directories (-x avoids filesystems other than the requested one, "." = search from where you are) 2du -sh * # show the total without listing the sub-directories (h = human readable) 3du -max . | sort -n | tail -30 # the 30 largest files/directories 4du -ks * | sort -n # size in kilobytes of all files and directories, where you are 5du -hsc * | sort -h # from smallest to largest 6ls -lrS # list files by size (in bytes) - note: ls -l does not give the true value contained in a directory 7du -ch /dir/ # size of the directories contained in /dir/ (with suffix) then the total Reduce / Truncate a file 1perl -e 'truncate "wanted_file", 100000' 2truncate -s 0 /ftpusers/ftp.upload.log File deleted but space still held by a process 1lsof +aL1 # "+L1" selects open files that have been "unlinked" (deleted but still open) 2lsof -nP | grep '(deleted)' 3find /proc/*/fd -type f -links 0 -exec ls -lrt {} \; # [SunOS] There are two solutions:
๐Ÿ’พ Backup & Sync
๐Ÿ’พ Backup & Sync
Rsync The classic formula 1rsync -arv --info=progress2 photo backup_photo a = archive โ€” preserves permissions (owner, group), times, symbolic links and devices. r = recursive โ€” copies directories and sub-directories. v = verbose โ€” prints what is being copied. Examples 1rsync -apvz --stats --update --exclude gsast/olap_cubes --exclude gsast/param user@server-src:/export/ user@server-dest:/home/ 2rsync -av -e ssh root@192.168.1.10:/backup/DUMP/* . 3rsync -azp --stats root@oracle-src:/ec/sw/oracle/client/product/12.2.0.1/network/mesg/ ~/mesg/ 4rsync -azp /home/user/mesg/ root@oracle-dest.example.com:/ec/sw/oracle/client/product/12.2.0.1/network/mesg/ 5 6ssh root@oracle-dest.example.com "ls -lrt /ec/sw/oracle/client/product/12.2.0.1/network/mesg/" 7ssh root@oracle-dest.example.com "chown oracle:dc_dba /ec/sw/oracle/client/product/12.2.0.1/network/mesg/*" 8 9rsync -aS --delete --rsh /export/home backup-host:/export/save Propagate deletions to the backup If you delete files in the source directory, rsync does not propagate the deletion to the backup directory unless you add the --delete option.
๐Ÿ—œ Compression
๐Ÿ—œ Compression
Zip / Unzip 1zip <archive.zip> <file1> <file2> # compress files. 2zip -r <archive.zip> <directory> # compress a directory. 3unzip archive_name.zip [-d directory] # decompress an archive. Gzip / Gunzip gzip is based on the Deflate algorithm (a combination of the LZ77 and Huffman algorithms). 1gzip -l # show the size of the uncompressed file. 2gzip <file> # compress. 3gunzip <file.gz> | gzip -d <file.gz> # decompress. 4gzip -9 <my_file> # compress a file optimally. 5gzip -c <file1> <file2> > compressed_file.gz # compress several files into a single one. Bzip2 / Bunzip2 bzip2 is an alternative to gzip, more efficient but slower.
๐ŸชŸ Tmux
๐ŸชŸ Tmux
Tmux git clone https://github.com/tmux-plugins/tmux-logging.git Command line tmux new -s my_session : Create a new session. tmux attach : Attach to the last used session. tmux attach -t X : Attach to the tmux session with number X. tmux ls : List active tmux sessions. tmux split-window -dh "!!" : Run a command in a separate pane. tmux source-file ~/.tmux.conf : Reload config. Basic Commands with key-bindings C-b w : List sessions/panes. C-b x : Close pane or session.
๐Ÿ›ฐ๏ธ Satellite
๐Ÿ—’๏ธ Sessions
๐Ÿ—’๏ธ Sessions
Register your session Useful to keep a trace, or to document and share what has been done. script : save all commands and results in a “typescript” file. script -a : append to an existing “typescript” file (otherwise erase the previous one). exit : to stop the session. asciinema : save the terminal session as a video. For RHEL - something like Tlog exists and can be configured and centralised with Rsyslog.
๐Ÿ–ฅ๏ธ GUI
Terraform
Terraform
Validate Terraform code 1dirs -c 2for DIR in $(find ./examples -type d); do 3 pushd $DIR 4 terraform init -backend=false 5 terraform fmt -check 6 terraform validate 7 popd 8 done Execute Terraform 1export DO_PAT="dop_v1_xxxxxxxxxxxxxxxx" 2doctl auth init --context rkub 3 4# inside a dir with a tf file 5terraform init 6terraform validate 7terraform plan -var "do_token=${DO_PAT}" 8terraform apply -var "do_token=${DO_PAT}" -auto-approve 9 10# clean apply 11terraform plan -out=infra.tfplan -var "do_token=${DO_PAT}" 12terraform apply infra.tfplan 13 14# Control 15terraform show terraform.tfstate 16 17# Destroy 18terraform plan -destroy -out=terraform.tfplan -var "do_token=${DO_PAT}" 19terraform apply terraform.tfplan Connect to server getting the ip with terraform command: 1ssh root@$(terraform output -json ip_address_workers | jq -r '.[0]') -i .key Troubleshoot some terraform Check the schema of a Resource (for example libvirt_domain from provider multani/libvirt ) 1terraform providers schema -json| jq '.provider_schemas["registry.terraform.io/multani/libvirt"].resource_schemas["libvirt_domain"].block.attributes | keys' 2[ 3 "arch", 4 "autostart", 5 "cloudinit", 6 "cmdline", 7 "coreos_ignition", 8 "cpu", 9 "description", 10 "disk", 11 "id", 12... 13] Then check what is expected: 1terraform providers schema -json| jq '.provider_schemas["registry.terraform.io/multani/libvirt"].resource_schemas["libvirt_domain"].block.attributes.cpu' 2["libvirt_domain"].block.attributes.cpu' 3{ 4 "type": [ 5 "map", 6 "string" 7 ], 8 "description_kind": "plain", Work with yaml in terraform Two possibilities:
S3 blockstorage
S3 blockstorage
S3cmd command S3cmd is a tool to handle blockstorage S3 type. Install the command 1# Ubuntu install 2sudo apt-get install s3cmd 3 4# Redhat install 5sudo dnf install s3cmd 6 7# or from sources 8wget https://sourceforge.net/projects/s3tools/files/s3cmd/2.2.0/s3cmd-2.2.0.tar.gz 9tar xzf s3cmd-2.2.0.tar.gz 10cd s3cmd-2.2.0 11sudo python3 setup.py install Configure it From Cloud providers (for example DO): Log in to the DigitalOcean Control Panel. Navigate to API > Spaces Access Keys and generate a new key pair.
Pull
Pull
Test locally a playbook 1ansible-pull -U https://github.com/MozeBaltyk/Okub.git ./playbooks/tasks/provision.yml Inside a cloud-init 1#cloud-config 2timezone: ${timezone} 3 4packages: 5 - qemu-guest-agent 6 - git 7 8package_update: true 9package_upgrade: true 10 11 12## Test 1 13ansible: 14 install_method: pip 15 package_name: ansible-core 16 run_user: ansible 17 galaxy: 18 actions: 19 - ["ansible-galaxy", "collection", "install", "community.general"] 20 - ["ansible-galaxy", "collection", "install", "ansible.posix"] 21 - ["ansible-galaxy", "collection", "install", "ansible.utils"] 22 pull: 23 playbook_name: ./playbooks/tasks/provision.yml 24 url: "https://github.com/MozeBaltyk/Okub.git" 25 26## Test 2 27ansible: 28 install_method: pip 29 package_name: ansible 30 #run_user only with install_method: pip 31 run_user: ansible 32 setup_controller: 33 repositories: 34 - path: /home/ansible/Okub 35 source: https://github.com/MozeBaltyk/Okub.git 36 run_ansible: 37 - playbook_dir: /home/ansible/Okub 38 playbook_name: ./playbooks/tasks/provision.yml 39######## Troubleshooting 1systemctl --failed 2systemctl list-jobs --after 3journalctl -e Checks user-data and config:
Parsing
Parsing
POO 1# Convert your json in object and put it in variable 2$a = Get-Content 'D:\temp\mytest.json' -raw | ConvertFrom-Json 3$a.update | % {if($_.name -eq 'test1'){$_.version=3.0}} 4 5$a | ConvertTo-Json -depth 32| set-content 'D:\temp\mytestBis.json' Example updating a XML 1#The file we want to change 2$xmlFilePath = "$MyPath\EXAMPLE\some.config" 3 4 # Read the XML file content 5 $xml = [xml](Get-Content $xmlFilePath) 6 7 $node = $xml.connectionStrings.add | where {$_.name -eq 'MetaData' -And $_.providerName -eq 'MySql.Data.MySqlClient'} 8 $node.connectionString = $AuditDB_Value 9 10 $node1 = $xml.connectionStrings.add | where {$_.name -eq 'Account'} 11 $node1.connectionString = $Account_Value 12 13 # Save the updated XML back to the file 14 $xml.Save($xmlFilePath) 15 16 Write-Host "$xmlFilePath Updated" Nested loop between a JSON and CSV 1# Read the JSON file and convert to a PowerShell object 2$jsonContent = Get-Content -Raw -Path ".\example.json" | ConvertFrom-Json 3 4# Read CSV and set a Header to determine the column 5$csvState = Import-CSV -Path .\referentials\states.csv -Header "ID", "VALUE" -Delimiter "`t" 6# Convert in object 7$csvState | ForEach-Object { $TableState[$_.ID] = $_.VALUE } 8 9# Loop through the Entities array and look for the state 10foreach ($item in $jsonContent.Entities) { 11 $stateValue = $item.State 12 13 # Compare the ID and stateValue then get the Value 14 $status = ($csvState | Where-Object { $_.'ID' -eq $stateValue }).VALUE 15 16 Write-Host "Status: $status" 17} Sources https://devblogs.microsoft.com/powershell-community/update-xml-files-using-powershell/
Mysql
Mysql
Example 1# Import values with connection details 2. .\values.ps1 3 4$scriptFilePath ="$MyPath\Install\MysqlBase\Script.sql" 5 6# Load the required DLL file (depend on your connector) 7[void][System.Reflection.Assembly]::LoadFrom("C:\Program Files (x86)\MySQL\MySQL Connector Net 8.0.23\Assemblies\v4.5.2\MySql.Data.dll") 8 9# Load in var the SQL script file 10$scriptContent = Get-Content -Path $scriptFilePath -Raw 11 12# Execute the modified SQL script 13$Connection = [MySql.Data.MySqlClient.MySqlConnection]@{ 14 ConnectionString = "server=$MysqlIP;uid=$MysqlUser;Port=3306;user id=$MysqlUser;pwd=$MysqlPassword;database=$MysqlDatabase;pooling=false;CharSet=utf8;SslMode=none" 15 } 16 $sql = New-Object MySql.Data.MySqlClient.MySqlCommand 17 $sql.Connection = $Connection 18 $sql.CommandText = $scriptContent 19 write-host $sql.CommandText 20 $Connection.Open() 21 $sql.ExecuteNonQuery() 22 $Connection.Close()
Inventory
Inventory
1ansible-inventory --list | jq -r 'map_values(select(.hosts != null and (.hosts | contains(["myhost"])))) | keys[]' 1kafka_host: "[{{ groups['KAFKA'] | map('extract', hostvars, 'inventory_hostname') | map('regex_replace', '^', '\"') | map('regex_replace', '\\\"', '\"') | map('regex_replace', '$', ':'+ kafka_port +'\"') | join(', ') }}]" 2 3elasticsearch_host: "{{ groups['ELASTICSEARCH'] | map('extract', hostvars, 'inventory_hostname') | map('regex_replace', '^', '\"') | map('regex_replace', '\\\"', '\"') | map('regex_replace', '$', ':'+ elasticsearch_port +'\"') | join(', ') }}"
Install
Install
Prerequisistes Check Compatibilty hardware: Oracle Linux Hardware Certification List (HCL) A minimum of two (2) KVM hosts and no more than seven (7). A fully-qualified domain name for your engine and host with forward and reverse lookup records set in the DNS. /var/tmp 10 GB space at least Prepared a shared-storage (nfs or iscsi) of at least 74 GB to be used as a data storage domain dedicated to the engine virtual machine. ISCSI need to be discovered before oVirt install.
๐Ÿ†” IDM
๐Ÿ†” IDM
Server Idm - Identity Manager prerequisites : repository configured NTP synchronize check config DHCP/DNS hostname -f == hostname acces to webui IDM : https://idm01.idm.example.com/ipa/ui/ 1yum install -y ipa-server ipa-server-dns 2 3ipa-server-install \ 4 --domain=example.com \ 5 --realm=EXAMPLE.COM \ 6 --ds-password=password \ 7 --admin-password=password \ 8 --hostname=classroom.example.com \ 9 --ip-address=172.25.0.254 \ 10 --reverse-zone=0.25.172.in-addr.arpa. \ 11 --forwarder=208.67.222.222 \ 12 --allow-zone-overlap \ 13 --setup-dns \ 14 --unattended Client link to IDM 1yum install -y ipa-client 2 3ipa-client-install --mkhomedir --enable-dns-updates --force-ntpd -p admin@EXAMPLE.COM --password='password' --force-join -U 4 5# Test login 6echo -n 'password' | kinit admin Script if DNS config is right for a IDM server 1sudo sh -c "cat <<EOF > ~/IdmZoneCheck.sh 2#!/bin/bash 3### IdM zone check ### 4# Check if the zone name is provided as a parameter # 5if [ -z "$1" ]; 6then 7 echo -e "Provide the zone name to be checked as a parameter!\n(ex: IdmZoneCheck.sh domain.local)" 8 exit 9fi 10clear 11echo -e "### IDM / TCP ###\n\n" 12echo -e "TCP / kerberos-master (SRV)" 13dig +short _kerberos-master._tcp.$1. SRV 14echo -e "_TCP / kerberos (SRV)" 15dig +short _kerberos._tcp.$1. SRV 16echo -e "_TCP / kpasswd (SRV)" 17dig +short _kpasswd._tcp.$1. SRV 18echo -e "_TCP / ldap (SRV)" 19dig +short _ldap._tcp.$1. SRV 20echo -e "\n### IDM / UDP ###\n\n" 21echo -e "_UDP / kerberos-master (SRV)" 22dig +short _kerberos-master._udp.$1. SRV 23echo -e "_UDP / kerberos (SRV)" 24dig +short _kerberos._udp.$1. SRV 25echo -e "_UCP / kpasswd (SRV)" 26dig +short _kpasswd._udp.$1. SRV 27echo -e "\n### IDM / MSDCS DC TCP ###\n\n" 28echo -e "_MSDCS / TCP / kerberos (SRV)" 29dig +short _kerberos._tcp.dc._msdcs.$1. SRV 30echo -e "_MSDCS / TCP / ldap (SRV)" 31dig +short _ldap._tcp.dc._msdcs.$1. SRV 32echo -e "\n### IDM / MSDCS DC UDP ###\n\n" 33echo -e "_MSDCS / UDP / kerberos (SRV)" 34dig +short _kerberos._udp.dc._msdcs.$1. SRV 35echo -e "\n### IDM / REALM ###\n\n" 36echo -e "REALM (TXT)" 37dig +short _kerberos.$1. TXT 38echo -e "\n### IDM / CA ###\n\n" 39echo -e "A / ipa-ca" 40dig +short ipa-ca.$1. A 41echo -e "\n### IDM / A ###\n\n" 42echo -e "A / $HOSTNAME" 43dig +short $HOSTNAME. A 44EOF Script usage : 1./IdmZoneCheck.sh idm.example.com
Gitlab
Gitlab
Glab CLI https://glab.readthedocs.io/en/latest/intro.html 1# add token 2glab auth login --hostname mygitlab.example.com 3# view fork of dep installer 4glab repo view mygitlab.example.com/copain/project 5# clone fork of dep installer 6glab repo clone mygitlab.example.com/copain/project Install 1Optimization 2puma['worker_processes'] = 16 3puma['worker_timeout'] = 60 4puma['min_threads'] = 1 5puma['max_threads'] = 4 6puma['per_worker_max_memory_mb'] = 2048 Certificats Generate CSR in /data/gitlab/csr/server_cert.cnf 1[req] 2default_bits = 2048 3distinguished_name = req_distinguished_name 4req_extensions = req_ext 5prompt = no 6 7[req_distinguished_name] 8C = PL 9ST = Poland 10L = Warsaw 11O = myOrg 12OU = DEV 13CN = gitlab.example.com 14 15[req_ext] 16subjectAltName = @alt_names 17 18[alt_names] 19DNS = gitlab.example.com 20IP = 192.168.01.01 1# Create CSR 2openssl req -new -newkey rsa:2048 -nodes -keyout gitlab.example.com.key -config /data/gitlab/csr/server_cert.cnf -out gitlab.example.com.csr 3 4openssl req -noout -text -in gitlab.example.com.csr 5 6# Sign your CSR with your PKI. If you PKI is a windows one, you should get back a .CER file. 7 8# check info: 9openssl x509 -text -in gitlab.example.com.cer -noout 1### push it in crt/key in Gitlab 2cp /tmp/gitlab.example.com.cer cert/gitlab.example.com.crt 3cp /tmp/gitlab.example.com.key cert/gitlab.example.com.key 4cp /tmp/gitlab.example.com.cer cert/192.168.01.01.crt 5cp /tmp/gitlab.example.com.key cert/192.168.01.01.key 6 7### push rootCA in gitlab 8cp /etc/pki/ca-trust/source/anchors/domain-issuing.crt /data/gitlab/config/trusted-certs/domain-issuing.crt 9cp /etc/pki/ca-trust/source/anchors/domain-rootca.crt /data/gitlab/config/trusted-certs/domain-rootca.crt 10 11### Reconfigure 12vi /data/gitlab/config/gitlab.rb 13docker exec gitlab bash -c 'update-ca-certificates' 14docker exec gitlab bash -c 'gitlab-ctl reconfigure' 15 16### Stop / Start 17docker stop gitlab 18docker rm gitlab 19docker run -d -p 5050:5050 -p 2289:22 -p 443:443 --restart=always \ 20-v /data/gitlab/config:/etc/gitlab \ 21-v /data/gitlab/logs:/var/log/gitlab \ 22-v /data/gitlab/data:/var/opt/gitlab \ 23-v /data/gitlab/cert:/etc/gitlab/ssl \ 24-v /data/gitlab/config/trusted-certs:/usr/local/share/ca-certificates \ 25--name gitlab gitlab/gitlab-ce:15.0.5-ce.0 Health-Checks 1docker exec gitlab bash -c 'gitlab-ctl status' 2docker exec -it gitlab gitlab-rake gitlab:check SANITIZE=true 3docker exec -it gitlab gitlab-rake gitlab:env:info Backup 1docker exec -it gitlab gitlab-rake gitlab:backup:create --trace 2 3#Alternate way to do it 4docker exec gitlab bash -c 'gitlab-backup create' 5docker exec gitlab bash -c 'gitlab-backup create SKIP=repositories' 6docker exec gitlab bash -c 'gitlab-backup create SKIP=registry' Restore from a Backup 1Restore 2gitlab-ctl reconfigure 3gitlab-ctl start 4gitlab-ctl stop unicorn 5gitlab-ctl stop sidekiq 6gitlab-ctl status 7ls -lart /var/opt/gitlab/backups 8 9docker exec -it gitlab gitlab-rake gitlab:backup:restore --trace 10docker exec -it gitlab gitlab-rake gitlab:backup:restore BACKUP=1537738690_2018_09_23_10.8.3 --trace 11 12Restart 13docker exec gitlab bash -c 'gitlab-ctl restart' Update Pre-checks before update sudo docker exec -it gitlab gitlab-rake gitlab:check sudo docker exec -it gitlab gitlab-rake gitlab:doctor:secrets
Github
Github
Get tag_name from latest 1export RKE_VERSION=$(curl -s https://update.rke2.io/v1-release/channels | jq -r '.data[] | select(.id=="stable") | .latest' | awk -F"+" '{print $1}'| sed 's/v//') 2export CERT_VERSION=$(curl -s https://api.github.com/repos/cert-manager/cert-manager/releases/latest | jq -r .tag_name) 3export RANCHER_VERSION=$(curl -s https://api.github.com/repos/rancher/rancher/releases/latest | jq -r .tag_name) 4export LONGHORN_VERSION=$(curl -s https://api.github.com/repos/longhorn/longhorn/releases/latest | jq -r .tag_name) 5export NEU_VERSION=$(curl -s https://api.github.com/repos/neuvector/neuvector-helm/releases/latest | jq -r .tag_name) Install gh 1# ubuntu 2type -p curl >/dev/null || (sudo apt update && sudo apt install curl -y) 3curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | sudo dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg \ 4&& sudo chmod go+r /usr/share/keyrings/githubcli-archive-keyring.gpg \ 5&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | sudo tee /etc/apt/sources.list.d/github-cli.list > /dev/null \ 6&& sudo apt update \ 7&& sudo apt install gh -y 8 9# Redhat 10sudo dnf install 'dnf-command(config-manager)' 11sudo dnf config-manager --add-repo https://cli.github.com/packages/rpm/gh-cli.repo 12sudo dnf install gh Autocompletions 1gh completion zsh > $ZSH/completions/_gh Create an ssh key ed Login 1gh auth login -p ssh -h GitHub.com -s read:project,delete:repo,repo,workflow -w 2 3gh auth status 4github.com 5 โœ“ Logged in to github.com as MorzeBaltyk ($HOME/.config/gh/hosts.yml) 6 โœ“ Git operations for github.com configured to use ssh protocol. 7 โœ“ Token: gho_************************************ 8 โœ“ Token scopes: delete_repo, gist, read:org, read:project, repo To use your key One way:
Gitea
Gitea
Prerequis - Firewalld activated, important otherwise the routing to the app is not working - Podman, jq installed Import image 1podman pull docker.io/gitea/gitea:1-rootless 2podman save docker.io/gitea/gitea:1-rootless -o gitea-rootless.tar 3podman load < gitea-rootless.tar Install cat /etc/systemd/system/container-gitea-app.service 1# container-gitea-app.service 2[Unit] 3Description=Podman container-gitea-app.service 4 5Wants=network.target 6After=network-online.target 7RequiresMountsFor=/var/lib/containers/storage /var/run/containers/storage 8 9[Service] 10Environment=PODMAN_SYSTEMD_UNIT=%n 11Restart=on-failure 12TimeoutStopSec=70 13PIDFile=%t/container-gitea-app.pid 14Type=forking 15 16ExecStartPre=/bin/rm -f %t/container-gitea-app.pid %t/container-gitea-app.ctr-id 17ExecStart=/usr/bin/podman container run \ 18 --conmon-pidfile %t/container-gitea-app.pid \ 19 --cidfile %t/container-gitea-app.ctr-id \ 20 --cgroups=no-conmon \ 21 --replace \ 22 --detach \ 23 --tty \ 24 --env DB_TYPE=sqlite3 \ 25 --env DB_HOST=gitea-db:3306 \ 26 --env DB_NAME=gitea \ 27 --env DB_USER=gitea \ 28 --env DB_PASSWD=9Oq6P9Tsm6j8J7c18Jxc \ 29 --volume gitea-data-volume:/var/lib/gitea:Z \ 30 --volume gitea-config-volume:/etc/gitea:Z \ 31 --network gitea-net \ 32 --publish 2222:2222 \ 33 --publish 3000:3000 \ 34 --label "io.containers.autoupdate=registry" \ 35 --name gitea-app \ 36 docker.io/gitea/gitea:1-rootless 37 38ExecStop=/usr/bin/podman container stop \ 39 --ignore \ 40 --cidfile %t/container-gitea-app.ctr-id \ 41 -t 10 42 43ExecStopPost=/usr/bin/podman container rm \ 44 --ignore \ 45 -f \ 46 --cidfile %t/container-gitea-app.ctr-id 47 48[Install] 49WantedBy=multi-user.target default.target Configuration inside /var/lib/containers/storage/volumes/gitea-config-volume/_data/app.ini
Git
Git
GIT is a distributed version control system that was created by Linus Torvalds, the mastermind of Linux itself. It was designed to be a superior version control system to those that were readily available, the two most common of these being CVS and Subversion (SVN). Whereas CVS and SVN use the Client/Server model for their systems, GIT operates a little differently. Instead of downloading a project, making changes, and uploading it back to the server, GIT makes the local machine act as a server. Tecmint
Collection
Collection
List 1ansible-galaxy collection list Install an Ansible Collection 1# From Ansible Galaxy official repo 2ansible-galaxy collection install community.general 3 4# From a tarball locally 5ansible-galaxy collection install ./community-general-6.0.0.tar.gz 6 7# From custom Repo 8ansible-galaxy collection install git+https://git.example.com/projects/namespace.collectionName.git 9ansible-galaxy collection install git+https://git.example.com/projects/namespace.collectionName,v1.0.2 10ansible-galaxy collection install git+https://git.example.com/namespace/collectionName.git 11 12# From a requirement.yml file 13ansible-galaxy collection install -r ./requirement.yaml Requirement file to install Ansible Collection 1collections: 2- name: kubernetes.core 3 4- source: https://gitlab.example.com/super-group/collector.git 5 type: git 6 version: "v1.0.6" 7 8- source: https://gitlab.ipolicedev.int/another-projects/plates.git 9 type: git
CEPH
Administration
Administration
Hosted-engine Administration Connect to VM hosted-engine with root and password setup during the install: 1# Generate a backup 2engine-backup --scope=all --mode=backup --file=/root/backup --log=/root/backuplog 3 4# Restore from a backup on Fresh install 5engine-backup --mode=restore --file=file_name --log=log_file_name --provision-db --restore-permissions 6engine-setup 7 8# Restore a backup on existing install 9engine-cleanup 10engine-backup --mode=restore --file=file_name --log=log_file_name --restore-permissions 11engine-setup host Administration Connect in ssh to the Host: 1# Pass a host in maintenance mode manually 2hosted-engine --vm-status 3hosted-engine --set-maintenance --mode=global 4hosted-engine --vm-status 5 6# Remove maintenance mode 7hosted-engine --set-maintenance --mode=none 8hosted-engine --vm-status 9 10# upgrade hosted-engine 11hosted-engine --set-maintenance --mode=none 12hosted-engine --vm-status 13engine-upgrade-check 14dnf update ovirt\*setup\* # update the setup package 15engine-setup # launch it to update the engine /!\ Connect individually to KVM Virtmanager does not work OVirt use libvirt but not like KVM do…
๐Ÿšฆ Gita
๐Ÿšฆ Gita
Presentation Gita is opensource project in python to handle a bit number of projects available: Here 1# Install 2pip3 install -U gita 3 4# add repo in gita 5gita add dcc/ssg/toolset 6gita add -r dcc/ssg # recursively add 7gita add -a dcc # resursively add and auto-group based on folder structure 8 9# create a group 10gita group add docs -n ccn 11 12# Checks 13gita ls 14gita ll -g 15gita group ls 16gita group ll 17gita st dcc 18 19# Use 20gita pull ccn 21gita push ccn 22 23gita freeze