Documentation about commands that should work on all Unix-like systems.
- The BIOS is started automatically and detects the peripherals.
- Loads the boot routine from the MBR (Master Boot Record) - it is the boot disk, located on the first sector of the hard disk.
- The MBR contains a loader that loads the "second stage loader": this is the "boot loader" specific to the system being loaded.
-> Linux uses LILO (Linux Loader) or GRUB (Grand Unified Bootloader).
- LILO loads the kernel into memory, decompresses it, and passes it the parameters.
- The kernel mounts the `/` filesystem (from there, the commands in `/sbin` and `/bin` are available).
- The kernel runs its first process: `init`.
LILO can offer several kernels as choices. The default choice: “Linux”.
/etc/lilo.conf : configuration of the kernel parameters.
/sbin/lilo : to write the new parameters to disk.
-> creates the /boot/map file, which contains the physical blocks where the boot program is located.
systemd replaces the SysV init system: services are managed with systemctl,
and runlevels map to targets.
1systemctl status <unit> # status of a service.
2systemctl start|stop|restart <unit> # run / stop / restart.
3systemctl enable|disable <unit> # start at boot (or not).
4systemctl isolate multi-user.target # equivalent of runlevel 3.
5systemctl set-default multi-user.target # change the default target.
6systemctl get-default
See the Runlevels & Shutdown page for the classic runlevel table.
| Solaris | Red Hat | Ubuntu / Debian | HP-UX | AIX | |
|---|---|---|---|---|---|
| Power down | shutdown -i5 -g0 -y | shutdown -h | shutdown -h | shutdown -h now | shutdown -F |
| Reboot | shutdown -i6 -g0 -y | shutdown -r | shutdown -r | shutdown -r now | shutdown -Fr |
| OK prompt | shutdown -i0 -g0 -y | โ | โ | โ | โ |
| Fast | reboot -- -r (reconfigure) | shutdown -f (no fsck) | shutdown -P (power off) | shutdown -F (force fsck) | โ |
| Force fsck | touch /reconfigure | touch /forcefsck | edit /etc/default/rcS โ FSCKFIX=yes | โ | โ |
| Tool | Solaris | Red Hat | Ubuntu / Debian | HP-UX | AIX |
|---|---|---|---|---|---|
halt | โ | โ | โ | โ | โ |
init | โ | โ | โ | โ | โ |
poweroff | โ | โ | โ | โ | โ |
reboot | โ | โ | โ | โ | โ |
shutdown | โ | โ | โ | โ | โ |
telinit | โ | โ | โ | โ | โ |
uadmin | โ | โ | โ | โ | โ |
| Level | Solaris | Red Hat | Ubuntu / Debian | HP-UX | AIX |
|---|---|---|---|---|---|
| 0 | shutdown | halt | halt | halt | reserved |
| 1 | single user | single user | single user | single user | reserved |
| 2 | n/a | multiuser (no networking) | multiuser (default) | multiuser (networking) | multiuser + NFS |
| 3 | multi-user | multiuser (networking) | same as 2 | multiuser + NFS + CDE GUI (default) | user defined |
| 4 | n/a | unused | same as 2 | multiuser + NFS + VUE GUI | user defined |
| 5 | power off | GUI | same as 2 | n/a | user defined |
| 6 | reboot | reboot | reboot | n/a | user defined |
| 7-9 | โ | โ | โ | โ | user defined |
initdefault line in vi /etc/inittab.vi /etc/event.d/rc-default.systemctl isolate multi-user.target (runlevel 3), systemctl isolate graphical.target (runlevel 5), systemctl set-default multi-user.target. See the Systemd page.cloud-init status --wait usefull for scripting, waiting cloud-init to finish before going to next step.
cloud-init status --long
1status: done
2extended_status: done
3boot_status_code: enabled-by-generator
4last_update: Thu, 01 Jan 1970 00:00:55 +0000
5detail: DataSourceNoCloud [seed=/dev/sr0]
6errors: []
7recoverable_errors: {}
sudo cloud-init analyze show 1-- Boot Record 01 --
2The total time elapsed since completing an event is printed after the "@" character.
3The time the event takes is printed after the "+" character.
4
5Starting stage: init-local
6|`->no cache found @00.00600s +00.00000s
7|`->found local data from DataSourceNoCloud @00.01500s +00.12600s
8Finished stage: (init-local) 00.75400 seconds
9
10Starting stage: init-network
11|`->restored from cache with run check: DataSourceNoCloud [seed=/dev/sr0] @04.21100s +00.00200s
12|`->setting up datasource @04.22800s +00.00000s
13|`->reading and applying user-data @04.23400s +00.00500s
14|`->reading and applying vendor-data @04.23900s +00.00000s
15|`->reading and applying vendor-data2 @04.23900s +00.00000s
16|`->activating datasource @04.27100s +00.00100s
17|`->config-seed_random ran successfully and took 0.000 seconds @04.29500s +00.00100s
18|`->config-write_files ran successfully and took 0.001 seconds @04.29600s +00.00100s
19|`->config-growpart ran successfully and took 0.562 seconds @04.29700s +00.56200s
20|`->config-resizefs ran successfully and took 0.193 seconds @04.86000s +00.19200s
21|`->config-mounts ran successfully and took 0.001 seconds @05.05200s +00.00100s
22|`->config-set_hostname ran successfully and took 0.004 seconds @05.05300s +00.00500s
23|`->config-update_hostname ran successfully and took 0.001 seconds @05.05800s +00.00100s
24|`->config-update_etc_hosts ran successfully and took 0.005 seconds @05.05900s +00.00500s
25|`->config-users_groups ran successfully and took 0.216 seconds @05.06400s +00.21600s
26|`->config-ssh ran successfully and took 0.404 seconds @05.28100s +00.40400s
27|`->config-set_passwords ran successfully and took 0.001 seconds @05.68500s +00.00200s
28Finished stage: (init-network) 01.50000 seconds
29
30Starting stage: modules-config
31|`->config-ssh_import_id ran successfully and took 0.001 seconds @07.43300s +00.00100s
32|`->config-locale ran successfully and took 0.003 seconds @07.43400s +00.00300s
33|`->config-grub_dpkg ran successfully and took 0.352 seconds @07.43700s +00.35200s
34|`->config-apt_configure ran successfully and took 0.049 seconds @07.79000s +00.04800s
35|`->config-timezone ran successfully and took 0.007 seconds @07.83900s +00.00700s
36|`->config-runcmd ran successfully and took 0.001 seconds @07.84600s +00.00100s
37|`->config-byobu ran successfully and took 0.000 seconds @07.84700s +00.00100s
38Finished stage: (modules-config) 00.45400 seconds
39
40Starting stage: modules-final
41|`->config-package_update_upgrade_install ran successfully and took 26.632 seconds @20.56700s +26.63300s
42|`->config-write_files_deferred ran successfully and took 0.001 seconds @47.20000s +00.00200s
43|`->config-reset_rmc ran successfully and took 0.000 seconds @47.20200s +00.00100s
44|`->config-scripts_vendor ran successfully and took 0.001 seconds @47.20300s +00.00000s
45|`->config-scripts_per_once ran successfully and took 0.000 seconds @47.20300s +00.00100s
46|`->config-scripts_per_boot ran successfully and took 0.000 seconds @47.20400s +00.00000s
47|`->config-scripts_per_instance ran successfully and took 0.000 seconds @47.20400s +00.00100s
48|`->config-scripts_user ran successfully and took 0.558 seconds @47.20500s +00.55800s
49|`->config-ssh_authkey_fingerprints ran successfully and took 0.005 seconds @47.76400s +00.00500s
50|`->config-keys_to_console ran successfully and took 0.054 seconds @47.76900s +00.05500s
51|`->config-install_hotplug ran successfully and took 0.001 seconds @47.82400s +00.00100s
52|`->config-final_message ran successfully and took 0.001 seconds @47.82500s +00.00100s
53Finished stage: (modules-final) 27.29600 seconds
Check the logs: sudo tail -n 50 /var/log/cloud-init-output.log
1# [RHEL] RootCA from DC need to be installed on host:
2cp my-domain-issuing.crt /etc/pki/ca-trust/source/anchors/my_domain_issuing.crt
3cp my-domain-rootca.crt /etc/pki/ca-trust/source/anchors/my_domain_rootca.crt
4update-ca-trust extract
5
6# [Ubuntu]
7sudo apt-get install -y ca-certificates
8sudo cp local-ca.crt /usr/local/share/ca-certificates
9sudo update-ca-certificates
Out-of-band management refers to accessing and controlling a server (console, power on/off, BIOS, remote media) through a dedicated management channel that is separate from the normal data path. “In-band” means going through the OS and its network interface; “out-of-band” uses an independent controller (a BMC โ Baseboard Management Controller) with its own network port, so it still works even when the OS is down, the machine is hung, or the network stack is broken. RSA is one vendor/technology family of out-of-band access.
To verify that an ISO image is good: compute its checksum with a sha1 or sha256 key, then compare it with the key published on the official website.
1sha256sum image.iso
2sha1sum image.iso
1logrotate -d /etc/logrotate.d/app # test a new configuration.
2reading config info for /data/log/app
3Handling 1 logs
4rotating pattern: /data/log/app after 1 days (10 rotations)
5empty log files are rotated, old logs are removed
Usage: logrotate [OPTION...] <configfile>
-d, --debug Don't do anything, just test (implies -v)
-f, --force Force file rotation
-m, --mail=command Command to send mail (instead of `/bin/mail')
-s, --state=statefile Path of state file
-v, --verbose Display messages during rotation
1ntpstat # see which NTP server we synchronise with, and whether the sync is good.
1synchronised to NTP server (192.168.1.12) at stratum 4
2 time correct to within 68 ms
3 polling server every 1024 s
1ntpq -p # see the state of the peers.
2ntpq -c peers
remote refid st t when poll reach delay offset jitter
==============================================================================
+192.168.1.11 192.168.2.4 4 u 259 1024 373 0.731 -0.980 0.557
*192.168.1.12 192.168.3.21 3 u 385 1024 377 0.773 0.146 0.365
192.168.4.255 .BCST. 16 u - 64 0 0.000 0.000 0.000
*) is the one being used.- are currently discarded by the server-selection algorithm.+ are possible synchronisation candidates.remote โ the server name.refid โ the parent server’s identifier.st โ the server’s stratum.t โ the server type.when โ seconds elapsed since the last contact.poll โ seconds between each contact.reach โ bitmask of successful contacts (octal): the server considers itself synchronised when reach reaches 177; a quality, stable connection shows 377.delay โ estimated round-trip time (ms) of the UDP packet.offset โ estimated difference between the peer’s clock and the internal clock.jitter โ dispersion of the reference values obtained from this peer.1service ntpd restart # or: systemctl restart ntpd
1cat /etc/ntp.conf # "server example.com" + restart ntpd + enable
2/var/log/ntpstats
Old service that synchronises NTP at boot (install the package first).
How to approach performance monitoring and tuning in Linux, and the various subsystems (and performance metrics) that need to be monitored.
On a very high level, the following four subsystems need to be monitored:
Four critical performance metrics for the CPU: context switch, run queue, CPU utilization, and load average.
0.75 1.70 2.10 indicates that the load is coming down (0.75 = last 1 minute, 1.70 = last 5 minutes, 2.10 = last 15 minutes).All four subsystems are interrelated. Just because you see a high reads/second, writes/second, or I/O wait, it does not mean the issue is with the I/O subsystem. It also depends on what the application is doing. In most cases, the performance issue is caused by the application running on the Linux system.
1zip <archive.zip> <file1> <file2> # compress files.
2zip -r <archive.zip> <directory> # compress a directory.
3unzip archive_name.zip [-d directory] # decompress an archive.
gzip is based on the Deflate algorithm (a combination of the LZ77 and Huffman algorithms).
1gzip -l # show the size of the uncompressed file.
2gzip <file> # compress.
3gunzip <file.gz> | gzip -d <file.gz> # decompress.
4gzip -9 <my_file> # compress a file optimally.
5gzip -c <file1> <file2> > compressed_file.gz # compress several files into a single one.
bzip2 is an alternative to gzip, more efficient but slower.
Preserve files permissions and ownership.
The Basics
1# Archive
2tar cvf my_archive.tar <file1> <file2> </dir/folder/>
3
4## Archive and compress with zstd everything in the current dir and push to /target/dir
5tar -I zstd -vcf archive.tar.zstd -C /target/dir .
6
7# Extract
8tar xvf my_archive.tar
9
10# Extract and push to target dir
11tar -zxvf new.tar.gz -C /target/dir
Other useful options โข t : list archive’s content. โข T : archive list given by a file. โข P : preserve absolute path (useful for backing up /etc). โข X : exclude. โข z : gunzip compression. โข j : bzip2 compression. โข J : lzma compression.
1rsync -arv --info=progress2 photo backup_photo
a = archive โ preserves permissions (owner, group), times, symbolic links and devices.r = recursive โ copies directories and sub-directories.v = verbose โ prints what is being copied.1rsync -apvz --stats --update --exclude gsast/olap_cubes --exclude gsast/param user@server-src:/export/ user@server-dest:/home/
2rsync -av -e ssh root@192.168.1.10:/backup/DUMP/* .
3rsync -azp --stats root@oracle-src:/ec/sw/oracle/client/product/12.2.0.1/network/mesg/ ~/mesg/
4rsync -azp /home/user/mesg/ root@oracle-dest.example.com:/ec/sw/oracle/client/product/12.2.0.1/network/mesg/
5
6ssh root@oracle-dest.example.com "ls -lrt /ec/sw/oracle/client/product/12.2.0.1/network/mesg/"
7ssh root@oracle-dest.example.com "chown oracle:dc_dba /ec/sw/oracle/client/product/12.2.0.1/network/mesg/*"
8
9rsync -aS --delete --rsh /export/home backup-host:/export/save
If you delete files in the source directory, rsync does not propagate the deletion to the backup directory unless you add the --delete option.
1ps -fp <pid> # find a process by its PID.
2pidof httpd # find the PIDs of httpd.
3pidstat -lp <pid> # process name with all its complete arguments.
4 # for a tomcat or weblogic process, you can split the arguments with: sed 's/ -/\n -/g'
5pidstat -C "mysql" # find a process by its name (gives the PID and CPU load).
1pstree -pu # the process tree with PID and user (if pstree is not installed, use the alternatives below).
1ps -ejH
2 PID PGID SID TTY TIME CMD
3 1 1 1 ? 00:00:37 init
411016 11009 11009 ? 00:00:00 sshd
511017 11017 11017 pts/12 00:00:00 bash
611125 11125 11017 pts/12 00:00:00 telnet
1ps axjf
2 PPID PID PGID SID TTY TPGID STAT UID TIME COMMAND
3 0 1 1 1 ? -1 Ss 0 0:37 init [5]
4 8617 10610 10610 10610 ? -1 Ss 0 0:00 \_ sshd: support [priv]
510610 10710 10610 10610 ? -1 S 5027 0:00 \_ sshd: support@notty
610710 10711 10711 10711 ? -1 Ss 5027 0:00 \_ sshd: support@internal-sftp-server
1ps faux
2USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
3root 1 0.0 0.0 10372 696 ? Ss Aug09 0:37 init [5]
4user1 4168 0.0 0.0 8728 968 ? Ss Aug24 0:00 | \_ /bin/bash -c perl /data/supports/scripts/SRAM_asr5k.pl &>/dev/null
5user1 4174 0.0 0.0 34068 5044 ? S Aug24 0:00 | \_ perl /data/supports/SRAM_asr5k.pl
6user1 4188 0.0 0.0 8728 984 ? S Aug24 0:00 | \_ sh -c grep -c SRAM /data/syslogCOLLECT/LTE_*/*/*.20160824.log
The /proc filesystem exposes per-process information:
pidstat reports the CPU, memory, I/O and context-switch activity of processes.
1# pidstat -w -p 3446 2 5
2Linux 3.10.0-123.13.2.el7.x86_64 (localhost.localdomain) 12/26/2014
3_x86_64_ (1 CPU)
407:23:38 AM UID PID cswch/s nvcswch/s Command
507:23:40 AM 0 3446 0.50 0.00 sshd
607:23:42 AM 0 3446 0.50 0.00 sshd
707:23:44 AM 0 3446 0.50 0.00 sshd
807:23:46 AM 0 3446 0.50 0.00 sshd
907:23:48 AM 0 3446 0.50 0.00 sshd
10Average: 0 3446 0.50 0.00 sshd
cswch/s : number of voluntary context switches the task made per second. (A voluntary context switch occurs when a task blocks because it requires a resource that is unavailable.)nvcswch/s : number of non-voluntary context switches the task made per second. (An involuntary context switch takes place when a task executes for the duration of its time slice and is then forced to relinquish the processor.)1pidstat -r -p <PID> 3600 72 # every hour, 72 times - practical for long-term monitoring.
2
3pidstat -r -p <PID> 50 12
407:26:44 PM PID minflt/s majflt/s VSZ RSS %MEM Command
507:27:34 PM 13775 1.64 0.00 34957320 18183312 55.30 java
minflt/s : number of minor faults the task has made per second โ those which did not require loading a memory page from disk.majflt/s : number of major faults the task has made per second โ those which required loading a memory page from disk.VSZ : Virtual Size โ the virtual memory usage of the entire task in kilobytes.RSS : Resident Set Size โ the non-swapped physical memory used by the task in kilobytes.1pidstat -d -p <PID> 50 12
pidstat -d reports I/O statistics (kernels 2.6.20 and later only). The following values are displayed:
1strace -tt -p 24503
2
3strace -o strace01.out -e open -f bash --login -i # see the files opened during a bash connection.
4 # -o redirects the output / -e filters the system calls / -f follows forks (child processes)
5
6strace -f <binary_script> 2> trace.log : stdout -> the binary command result, stderr -> the binary's system calls.
ltrace traces shared-library calls (like strace, but at the library-call level).
Source : http://www.glennklockwood.com/hpc-howtos/process-affinity.html
1sudo apt-get install util-linux # or: yum install util-linux
2taskset -c 1 script.sh # run script.sh on CPU number 1 (-c: CPU, -p: PID)
3taskset -c 1,2,3 script.sh # give it several CPUs.
1numactl --cpunodebind=0 simulation.x
2numactl --cpunodebind=0 --membind=0 simulation.x
3numactl -C 0 -N 0 simulation.x
4numactl -C +0,1,2,3 simulation.x # similar to taskset
5numactl -H # see which memory corresponds to a CPU
Note: with numactl, unlike taskset, you cannot change the CPU affinity of a process on the fly.
1at : schedule a task to run at a later time (/!\ it executes what you give it on stdin).
2 ex : at 18:22 < "date; ps -ef | wc -l"
3 or : at now + 5 hours then type your commands then ctrl + d
4at -q a 16:05 tomorrow : -q defines the queue (a-z), i.e. the priority.
5at -c <job_number> : see the context and the commands of the task.
6atq : list the pending jobs (= at -l).
7atrm : delete a job.
8
9batch : schedule a task when the load average is below a threshold.
1jobs -l : list the running tasks.
1# Nohup in series
2for i in {1..6}; do echo "nohup sh -c \"shred -vfz -n 3 /dev/cciss/c0d${i} > nohup${i}.out 2>&1 \" &" ; done | bash
Three points to remember:
/etc/crontab : the daemon’s configuration file, defining the default behaviour of crond (SHELL, MAILTO, etc.)./etc/cron.d/... : system crontab (used by admins)./var/spool/cron/root : crontab per user.1MAILTO="admin@example.com"
2* * * * * root /usr/local/sbin/mycommand.sh > /dev/null 2>&1
/etc/anacrontab : file that runs, via the run-parts command, /etc/cron.daily, /etc/cron.weekly, /etc/cron.monthly./etc/cron.d/0hourly : exception, runs /etc/cron.hourly via run-parts, checking the last run of the task in /var/spool/anacron/....Exactly the last day of each month:
Change the root directory of a command or a process, and its children.
1# create the "jail" directory
2J=$HOME/jail
3mkdir -p $J
4mkdir -p $J/{bin,lib64,lib}
5cd $J
6
7# copy the binaries and their libraries into the jail
8cp -v /bin/{bash,ls} $J/bin
9
10list="$(ldd /bin/bash | egrep -o '/lib.*\.[0-9]')"
11for i in $list; do cp -v "$i" "${J}${i}"; done
12
13list="$(ldd /bin/ls | egrep -o '/lib.*\.[0-9]')"
14for i in $list; do cp -v "$i" "${J}${i}"; done
15
16# enter the jail
17sudo chroot $J /bin/bash
1locate <pattern> # find a directory or file quickly (uses an index); a brand-new file won't be found.
2updatedb # update the locate index.
1view <file> # opens a read-only vi view (preferred if you just want to search/view).
2gzcat / zcat <file.gz> # read a gzipped file.
1stat </my/file> # all info about a file (inode, creation, modification, access dates, etc.).
2stat -f <FS> # info about a filesystem.
3stat -c%s $LOGFILE # [scripting] get a precise value (size, modification date, etc.).
1grep -w 'xyz' # match the whole word.
2grep -x 'Hello, world!' # the whole line must match.
3grep -c <pattern> # count the matching lines.
4grep -l "ERROR:" *.log # search all .log files, list the files that match.
5grep -L <pattern> # inverse: list the files that do NOT match.
6grep -f <patternfile> <file> # apply the patterns read from patternfile.
7grep -i <pattern> # ignore case.
8grep -v <pattern> # return the lines that do NOT match.
9grep -m x <pattern> # stop after x matching lines.
10grep -n <pattern> # show the line number.
11grep -q <pattern> # quiet: exit 0 if found, 1 (or 2) otherwise (for scripting).
12grep -s <pattern> # suppress permission/inexistent-file error messages.
13grep -H <pattern> # show the filename next to each matching line.
14grep -h <pattern> # do not show the filename (default behaviour).
15grep -A x <pattern> # also show x lines After.
16grep -B x <pattern> # also show x lines Before.
17grep -C x <pattern> # show x lines of context (A + B).
18grep -a <pattern> <binary> # search a binary file as if it were text.
1egrep = grep -E # for complex regular expressions.
1# extract the 3rd field, then cut:
2cat file | grep /u01/grid/19c | awk '{print $3}' | cut -f2 -d'"'
3# is equivalent to:
4cat file | grep -o /u01/grid/19c
On a Unix machine, the system logs are in /var/log/messages (or /var/adm/messages on Solaris). This is where you find the errors, with log rotation.
| Linux | Solaris | HP-UX | AIX | BSD | |
|---|---|---|---|---|---|
| location | /var/log/messages, /var/log/secure, /var/log/boot.log | /var/adm/messages | /var/adm/syslog/mail.log, /var/adm/syslog/syslog.log | /tmp or none | /var/log/syslog |
| Type | Linux | Solaris | HP-UX | AIX |
|---|---|---|---|---|
| current logins | /var/run/utmp | /var/adm/utmpx | /var/adm/utmp | /etc/utmp |
| login history | /var/log/wtmp | /var/adm/wtmpx | /var/adm/wtmp | /var/adm/wtmp |
| process accounting | /var/log/pacct | /var/adm/pacct | /var/adm/pacct | /var/adm/pacct |
| Linux | Solaris | HP-UX | AIX | |
|---|---|---|---|---|
| failed logins | /var/log/btmp, /var/log/messages | /var/adm/loginlog, /var/adm/sulog | /var/adm/sulog | /etc/security/failedlogin |
1# today's logs
2grep "$(date '+%b %d')" /var/log/messages
3
4# disk errors (nawk: print the last field of the "Error Block" lines)
5nawk '/Error Block/{print $NF}' /var/adm/messages* | sort | uniq
6
7# find the IPs in a log, sort them and remove the duplicates
8cat /var/log/maillog | grep -Eo '([0-9]{1,3}\.){3}[0-9]{1,3}' | sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4 | uniq
1# ping a list of servers
2for ip in $(awk '/192.168.45/ {print $1}' /etc/hosts); do ping -c 1 $ip; done
1for vm in vm{1..27}; do ssh -q $vm "hostname; free; sar -r 3 3"; done
/var/log/auth.log โ SSH connection logs. Check that there are not too many failed connections (a sign of an intrusion attempt).
fuser: 1fuser -m </dir or /files> # Find process blocking/using this directory or files.
2fuser -cu </dir or /files> # Same as above but add the user
3fuser -kcu </dir or /files> # Kill process
4fuser -v -k -HUP -i ./ # Send HUP signal to process
5
6# Output will send you <PID + letter>, here is the meaning:
7# c current directory.
8# e executable being run.
9# f open file. (omitted in default display mode).
10# F open file for writing. (omitted in default display mode).
11# r root directory.
12# m mmap'ed file or shared library.
lsof ( = list open file):1lsof +D /var/log # Find all files blocked with the process and user.
2lsof -a +L1 <mountpoint> # Process blocking a FS.
3lsof -c ssh -c init # Find files open by thoses processes.
4lsof -p 1753 # Find files open by PID process.
5lsof -u root # Find files open by user.
6lsof -u ^user # Find files open by user except this one.
7kill -9 `lsof -t -u toto` # kill user's processes. (option -t output only PID).
1#When you have no fuser or lsof:
2find /proc/*/fd -type f -links 0 -exec ls -lrt {} \;
fuser)1fuser -d /tmp # see the processes using the /tmp directory (AIX)
2fuser -c /your_FS # all processes with an open file in the filesystem (AIX)
3fuser -cu /dev/vg01/lvol5 # also search with a filesystem or an LV
-c == -m ; -u also shows the process user.fuser -kcu.For detecting deleted-but-still-open files (lsof +L1) and freeing the held space, see the Disk Cleanup page.
1last # the last user connections to a server (based on /var/log/wtmp or btmp).
2ac -d # statistics of my connection time per day.
3ac -p <user> # the connection time of all users (or of a specific user).
4finger # who is connected (-l to also see mails and plans of all users).
5w # who is connected, doing what, and how much CPU they use.
6who # who is connected (-u for more info: PID, etc.).
7who am i # with which login I am connected.
8id -a # all info about the user I'm connected as (more precise than "who am i").
9logname # the login name of the current account.
1last reboot # see all the reboots that took place.
2uptime # see how long the server has been up + the load average.
3lslogins -L # also shows whether a user shutdown/rebooted the machine.
1diff <file1> <file2> # -w to ignore whitespace.
2colordiff <file1> <file2> # colourised diff.
3wdiff <file1> <file2> # word diff: [โ โ] replaced word, {+ +} added word.
4vimdiff <file1> <file2> # open both files in vim (blue = entirely different lines, red = partially different).
5fgrep -f <list> <file> # compare two lists (e.g. of hosts).
1diff -W200 -y <(unzip -vqq file1.jar | awk '{ if ($1 > 0) {printf("%s\t%s\n", $1, $8)}}' | sort -k2) <(unzip -vqq file2.jar | awk '{ if ($1 > 0) {printf("%s\t%s\n", $1, $8)}}' | sort -k2)
1# in crontab or tmux session - take every hour a track of the memory usage
2for i in {1..24} ; do echo -n "===================== " ; date ; free -m ; top -b -n1 | head -n 15 ; sleep 3600; done >> /var/log/SYSADM/memory.log &
1find . -type f -mtime +150 -exec ls -lrt {} \; | more
2find . -maxdepth 1 -name "*.log" -mtime +10 -exec ls -lrt {} \;
3find . -mtime +150 -exec rm -f {} \;
The find loop is cheaper than a shell loop (for ...). You can make it even more efficient by batching the rm calls:
1find . -type f -print -exec rm -- "{}" + # note the "+" instead of the usual "\;"
1du -max . # list all the FS sub-directories (-x avoids filesystems other than the requested one, "." = search from where you are)
2du -sh * # show the total without listing the sub-directories (h = human readable)
3du -max . | sort -n | tail -30 # the 30 largest files/directories
4du -ks * | sort -n # size in kilobytes of all files and directories, where you are
5du -hsc * | sort -h # from smallest to largest
6ls -lrS # list files by size (in bytes) - note: ls -l does not give the true value contained in a directory
7du -ch /dir/ # size of the directories contained in /dir/ (with suffix) then the total
1perl -e 'truncate "wanted_file", 100000'
2truncate -s 0 /ftpusers/ftp.upload.log
1lsof +aL1 # "+L1" selects open files that have been "unlinked" (deleted but still open)
2lsof -nP | grep '(deleted)'
3find /proc/*/fd -type f -links 0 -exec ls -lrt {} \; # [SunOS]
There are two solutions:
ext4 : the most widespread on GNU/Linux (derived from ext2 and ext3). It is journaled, meaning it records write operations to guarantee data integrity in case of an abrupt disk stop. It can also handle volumes up to 1 EiB (1024 PiB), and allows pre-allocating a contiguous area for a file to minimize fragmentation. Use this filesystem if you want to be able to read data back from macOS or Windows.
1yum install iscsi-initiator-utils
2
3#Checks
4iscsiadm -m session -P 0 # get the target name
5iscsiadm -m session -P 3 | grep "Target: iqn\|Attached scsi disk\|Current Portal"
6
7# Discover and mount ISCSI disk
8iscsiadm -m discovery -t st -p 192.168.1.112
9iscsiadm --mode discovery --type sendtargets --portal 192.168.1.112
10
11# Login
12iscsiadm -m node -T iqn.1992-04.com.emc:cx.ckm00192201413.b0 -l
13iscsiadm -m node -T iqn.1992-04.com.emc:cx.ckm00192201413.b1 -l
14iscsiadm -m node -T iqn.1992-04.com.emc:cx.ckm00192201413.a1 -l
15iscsiadm -m node -T iqn.1992-04.com.emc:cx.ckm00192201413.a0 -l
16
17# Enable/Start service
18systemctl enable iscsid iscsi && systemctl stop iscsid iscsi && systemctl start iscsid iscsi
1for BUS in /sys/class/scsi_host/host*/scan; do echo "- - -" > ${BUS} ; done
2
3sudo sh -c 'for BUS in /sys/class/scsi_host/host*/scan; do echo "- - -" > ${BUS} ; done '
Partition your FS
list of component:
LVM2 use a new driver, the device-mapper allow the us of diskยดs sectors in different targets: - linear (most used in LVM). - stripped (stripped on several disks) - error (all I/O are consider in errors) - snapshot (allow snapshot async)
1lvs --all --segments -o +devices
2server_xplore_col1 vgdata -wi-ao---- 21 striped 1.07t /dev/md2(40229),/dev/md3(40229),/dev/md4(40229),/dev/md5(40229),โฆ
3server_xplore_col2 vgdata -wi-ao---- 1 linear 219.87g /dev/md48(0)
1# Summary
2pvs
3vgs
4lvs
5
6# Scanner
7pvscan
8vgscan
9lvscan
10
11# Details info
12pvdisplay [sda]
13pvdisplay -m /dev/emcpowerd1
14vgdisplay [vg_root]
15lvdisplay [/dev/vg_root/lv_usr]
16
17# Summary details
18lvmdiskscan
19 /dev/sda1 [ 600.00 MiB]
20 /dev/sda2 [ 1.00 GiB]
21 /dev/sda3 [ 38.30 GiB] LVM physical volume
22 /dev/sdb1 [ <100.00 GiB] LVM physical volume
23 /dev/sdc1 [ <50.00 GiB] LVM physical volume
24 /dev/sdj [ 20.00 GiB]
25 1 disk
26 2 partitions
27 0 LVM physical volume whole disks
28 3 LVM physical volumes
1parted /dev/sda resizepart 3 100%
2udevadm settle
3pvresize /dev/sda3
4
5# Extend a XFS to a fixe size
6lvextend -L 30G /dev/vg00/var
7xfs_growfs /dev/vg00/var
8
9# Add some space to a ext4 FS
10lvextend -L +10G /dev/vg00/var
11resize2fs /dev/vg00/var
12
13# Extend to a pourcentage and resize automaticly whatever is the FS type.
14lvextend -l +100%FREE /dev/vg00/var -r
1parted /dev/sdb mklabel gpt mkpart primary 1 100% set 1 lvm on
2udevadm settle
3pvcreate /dev/sdb1
4vgcreate vg01 /dev/sdb1
5lvcreate -n lv_data -l 100%FREE vg01
6
7# Create a XFS
8mkfs.xfs /dev/vg01/lv_data
9mkdir /data
10echo "/dev/mapper/vg01-lv_data /data xfs defaults 0 0" >> /etc/fstab
11mount -a
12
13# Create an ext4
14mkfs.ext4 /dev/vg01/lv_data
15mkdir /data
16echo "/dev/mapper/vg01-lv_data /data ext4 defaults 0 0" >> /etc/fstab
17mount -a
1swapoff -v /dev/dm-1
2lvremove /dev/vg00/swap
3vi /etc/fstab
4vi /etc/default/grub
5grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg
6grubby --remove-args "rd.lvm.lv=vg00/swap" --update-kernel /boot/vmlinuz-3.10.0-1160.71.1.el7.x86_64
7grubby --remove-args "rd.lvm.lv=vg00swap" --update-kernel /boot/vmlinuz-3.10.0-1160.el7.x86_64
8grubby --remove-args "rd.lvm.lv=vg00/swap" --update-kernel /boot/vmlinuz-0-rescue-cd2525c8417d4f798a7e6c371121ef34
9echo "vm.swappiness = 0" >> /etc/sysctl.conf
10sysctl -p
1# #n case of crash, just relaunch pvmove without arguments
2pvmove /dev/emcpowerd1 /dev/emcpowerc1
3
4# Remove PV from a VG
5vgreduce /dev/emcpowerd1 vg01
6
7# Remove all unused PV from VG01
8vgreduce -a vg01
9
10# remove all PV
11pvremove /dev/emcpowerd1
/var even if doesn’t want:1lvchange -ay --ignorelockingfailure --sysinit vgroot/var
1# VG rename
2vgrename
3
4# LV rename
5lvrename
6
7# PV does not need to be rename
Even if in the past I was using partition MS-DOS disklabel or GPT disklabel for PV, I prefer now to use directly LVM on the main block device. There is no reason to use 2 disklabels, unless you have a very specific use case (like disk with boot sector and boot partition).
mdadm (multiple devices admin) is software solution to manage RAID.
It allow:
/dev/sdb, /dev/sdc) or (/dev/sdb1, /dev/sdc1)raidtools1# View real-time information about your md devices
2cat /proc/mdstat
3
4# Monitor for failed disks (indicated by "(F)" next to the disk)
5watch cat /proc/mdstat
1# Display details about the RAID array (replace /dev/md0 with your array)
2mdadm --detail /dev/md0
3
4# Examine RAID disks for information (not volume) similar to --detail
5mdadm --examine /dev/sd*
The conf file /etc/mdadm.conf does not exist by default and need to be created once you finish your install.
This file is required for the autobuild at boot.
1yum install device-mapper-multipath
vim /etc/multipath.conf:1defaults {
2user_friendly_names yes
3path_grouping_policy multibus
4}
1multipaths {
2 multipath {
3 wwid "36000d310004142000000000000000f23"
4 alias oralog1
5 }
1 devices {
2 device {
3 vendor "DGC"
4 product ".*"
5 product_blacklist "LUNZ"
6 :
7 path_checker emc_clariion ### Rev 47 alua
8 hardware_handler "1 alua" ### modified for alua
9 prio alua ### modified for alua
10 :
11 }
12 }
Checks config with: multipathd show config |more
NFS vs iscsi
Concurrent access to a block device like iSCSI is not possible with standard file systems. You’ll need a shared disk filesystem (like GFS or OCSFS) to allow this, but in most cases the easiest solution would be to just use a network share (via SMB/CIFS or NFS) if this is sufficient for your application.
1# check partion
2parted -l /dev/sda
3fdisk -l
4
5# check partition - visible before the mkfs
6ls /sys/sda/sda*
7ls /dev/sd*
8
9# give partition after the mkfs or pvcreate
10blkid
11blkid -o list
12
13# summary about the disks, partitions, FS and LVM
14lsblk
15lsblk -f
in script mode
1# with fdisk
2printf "n\np\n1\n\n\nt\n8e\nw\n" | sudo fdisk "/dev/sdb"
3
4# with parted
5sudo parted /dev/sdb mklabel gpt mkpart primary 1 100% set 1 lvm on
Gparted : interface graphique (ce base sur parted un utilitaire GNU - Table GPT)
First Install samba and samba-client (for debug + test)
/etc/samba/smb.conf1[home]
2Workgroup=WORKGROUP (le grp par defaul sur windows)
3Hosts allow = ...
4[shared]
5browseable = yes
6path = /shared
7valid users = user01, @un_group_au_choix
8writable = yes
9passdb backend = tdbsam #passwords are stored in the /var/lib/samba/private/passdb.tdb file.
testparm
/usr/bin/testparm -s /etc/samba/smb.conf
smbclient -L \192.168.56.102 -U test : list all samba shares available
smbclient //192.168.56.102/sharedrepo -U test : connect to the share
pdbedit -L : list user smb (better than smbclient)
S.M.A.R.T. is a technology that allows you to monitor and analyze the health and performance of your hard drives. It provides valuable information about the status of your storage devices. Here are some useful commands and tips for using S.M.A.R.T. with smartctl:
To display S.M.A.R.T. information for a specific drive, you can use the following command:
1smartctl -a /dev/sda
This command will show all available S.M.A.R.T. data for the /dev/sda drive.
SSHFS mounts a remote filesystem on your local filesystem through an SSH connection, all with user rights. The advantage is being able to manipulate remote data with any file manager (Nautilus, Konqueror, ROX, or even the command line).
- Prerequisites: administrator rights, ethernet connection, installation of FUSE and the SSHFS package.
- SSHFS users must belong to the `fuse` group.
Note: FUSE allows a user to mount a filesystem themselves. Normally, mounting a filesystem requires being an administrator, or having it pre-approved in /etc/fstab with hard-coded information.
1# Get the state
2firewall-cmd --state
3systemctl status firewalld
4
5# Get infos
6firewall-cmd --get-default-zone
7firewall-cmd --get-active-zones
8firewall-cmd --get-zones
9firewall-cmd --set-default-zone=home
10
11firewall-cmd --permanent --zone=FedoraWorkstation --add-source=00:FF:B0:CB:30:0A
12firewall-cmd --permanent --zone=FedoraWorkstation --add-service=ssh
13
14firewall-cmd --get-log-denied
15firewall-cmd --set-log-denied=<all, unicast, broadcast, multicast, or off>
1#Remove
2firewall-cmd --zone=public --add-service=ftp --permanent
3firewall-cmd --zone=public --remove-service=ftp --permanent
4firewall-cmd --zone=public --remove-port=53/tcp --permanent
5firewall-cmd --zone=public --list-services
6
7# Add
8firewall-cmd --zone=public --new-service=portal --permanent
9firewall-cmd --zone=public --service=portal --add-port=8080/tcp --permanent
10firewall-cmd --zone=public --service=portal --add-port=8443/tcp --permanent
11firewall-cmd --zone=public --add-service=portal --permanent
12firewall-cmd --reload
13
14firewall-cmd --zone=public --new-service=k3s-server --permanent
15firewall-cmd --zone=public --service=k3s-server --add-port=443/tcp --permanent
16firewall-cmd --zone=public --service=k3s-server --add-port=6443/tcp --permanent
17firewall-cmd --zone=public --service=k3s-server --add-port=8472/udp --permanent
18firewall-cmd --zone=public --service=k3s-server --add-port=10250/tcp --permanent
19firewall-cmd --zone=public --add-service=k3s-server --permanent
20firewall-cmd --reload
21
22firewall-cmd --zone=public --new-service=quay --permanent
23firewall-cmd --zone=public --service=quay --add-port=8443/tcp --permanent
24firewall-cmd --zone=public --add-service=quay --permanent
25firewall-cmd --reload
26
27firewall-cmd --get-services # It's also possible to add a service from list
28firewall-cmd --runtime-to-permanent
1for s in `firewall-cmd --list-services`; do echo $s; firewall-cmd --permanent --service "$s" --get-ports; done;
2
3sudo sh -c 'for s in `firewall-cmd --list-services`; do echo $s; firewall-cmd --permanent --service "$s" --get-ports; done;'
4ssh
522/tcp
6dhcpv6-client
7546/udp
1firewall-cmd --info-service cfrm-IC
2cfrm-IC
3 ports: 7780/tcp 8440/tcp 8443/tcp
4 protocols:
5 source-ports:
6 modules:
7 destination:
1firewall-cmd --list-all
2public (active)
3 target: default
4 icmp-block-inversion: no
5 interfaces: ens192
6 sources:
7 services: ssh dhcpv6-client https Oracle nimsoft
8 ports: 10050/tcp 1521/tcp
9 protocols:
10 masquerade: no
11 forward-ports:
12 source-ports:
13 icmp-blocks:
14 rich rules:
1firewall-cmd --zone=backup --list-all
1firewall-cmd --get-active-zones
2backup
3 interfaces: ens224
4public
5 interfaces: ens192
1ls /etc/firewalld/
2firewalld.conf helpers/ icmptypes/ ipsets/ lockdown-whitelist.xml services/ zones/
1firewall-cmd --get-ipset-types
2firewall-cmd --permanent --get-ipsets
3firewall-cmd --permanent --info-ipset=integration
4firewall-cmd --ipset=integration --get-entries
5
6firewall-cmd --permanent --new-ipset=test --type=hash:net
7firewall-cmd --ipset=local-blocklist --add-entry=103.133.104.0/23
1nmcli con show
2NAME UUID TYPE DEVICE
3ens192 4d0087a0-740a-4356-8d9e-f58b63fd180c ethernet ens192
4ens224 3dcb022b-62a2-4632-8b69-ab68e1901e3b ethernet ens224
5
6nmcli dev status
7DEVICE TYPE STATE CONNECTION
8ens192 ethernet connected ens192
9ens224 ethernet connected ens224
10ens256 ethernet connected ens256
11lo loopback unmanaged --
12
13# Get interfaces details :
14nmcli connection show ens192
15nmcli -p con show ens192
16
17# Get DNS settings in interface
18UUID=$(nmcli --get-values connection.uuid c show "cloud-init eth0")
19nmcli --get-values ipv4.dns c show $UUID
1nmcli connection add type ethernet mac "00:50:56:80:11:ff" ifname "ens224"
2nmcli connection add type ethernet mac "00:50:56:80:8a:0b" ifname "ens256"
1nmcli con load /etc/sysconfig/network-scripts/ifcfg-ens224
2nmcli con up ens192
1nmcli con mod enp1s0 +ipv4.addresses "192.168.122.11/24"
2ip addr del 10.10.10.36/24 dev ens160
3
4nmcli con reload # before to reapply
5nmcli device reapply ens224
6systemctl status network.service
7systemctl restart network.service
1UUID=$(nmcli --get-values connection.uuid c show "cloud-init eth0")
2DNS_LIST=$(nmcli --get-values ipv4.dns c show $UUID)
3nmcli conn modify "$UUID" ipv4.dns "${DNS_LIST} ${DNS_IP}"
4
5# /etc/resolved is managed by systemd-resolved
6sudo systemctl restart systemd-resolved
| File | Check command | Purpose |
|---|---|---|
/etc/passwd | pwck | user accounts |
/etc/group | grpck | groups |
/etc/shadow | โ | password hashes and aging |
/etc/gshadow | โ | group passwords |
/etc/skel | โ | files installed by default when a user is created |
1useradd -g <GID> -G <GID2> <user> # create a user in primary group GID (and supplementary group GID2).
2usermod <options> <user> # modify a user.
3userdel -r <user> # delete a user (and its home directory).
4groupadd / groupmod / groupdel # manage groups.
5
6id -a # show all info about the current user (UID, GUID, groups, etc.) - more precise than "who am i".
7sg <group> -c '<command>' # execute a command as a different group ID (to run scripts or write to a file with group rights).
1passwd -u <user> # unlock a user account.
2echo "password" | passwd --stdin <user> # scripted password change.
1chage -l <user> # see the expiration dates.
1# list the expiry of every account
2for account in $(cut -f1 -d: /etc/passwd); do
3 echo "ACCOUNT: $account , EXPIRES: $(chage -l $account | grep 'Account expires' | awk '{print $4, $5, $6}'), CHANGED: $(chage -l $account | grep 'Last password change' | awk '{print $5, $6, $7}')";
4done
1# change the aging info interactively
2chage <user>
To unlock an account, set “Last Password Change” to -1 in chage (or use passwd -u <user>).
The /etc/sudoers file contains the set of UNIX operating-system privileges that the local administrator has granted to UNIX users.
In no case should this file be edited directly with vi; it must be edited with visudo.
1sudo : execute a command as root.
2sudo su : become root and stay root.
3sudoers : file listing the commands allowed for certain users as the superuser (or another user).
4visudo -cs : strict syntax check of the sudoers file.
sudo -i is equivalent to su - in terms of rights.sudo -i, the user password is asked.su -, the root password is asked.1sudo -l -U <user>
1# "user" runs "sudo -u target All_the_commands"
2user server=(target) NOPASSWD: ALL
1Host_Alias LOAD_BALANCERS = server1,server2
2
3Cmnd_Alias SET_VIP = \
4/sbin/ip addr add 192.168.10.12/20 broadcast 192.168.15.255 dev eth0 label eth0\:0, \
5/sbin/ip addr del 192.168.10.12/20 dev eth0, \
6/sbin/arping -U -c 1 -I eth0 192.168.10.12
7
8loaduser LOAD_BALANCERS=(root) NOPASSWD: SET_VIP
9syncuser LOAD_BALANCERS=(root) NOPASSWD: SET_VIP
In /etc/pam.d, there is one PAM file per service.
Syntax: module_type control_flag path_to_module_agent
auth โ authentication.account โ account-based restrictions (validity, time of day, etc.).session โ things that run at login/logout.password โ password updates.required โ success needed; a failure is reported but only after the rest of the stack has run.requisite โ like required, but a failure returns immediately without running the rest of the stack.sufficient โ if this module succeeds, it is the last module tested in the stack.optional โ its result is only taken into account if no other module succeeded or failed.[value=action value=action2 ...] โ advanced control: map a module result to a specific action.Sample:
1kinit <user> # obtain a ticket.
2klist # list the tickets in the cache.
1systemctl status slapd # OpenLDAP server.
2systemctl status sssd # System Security Services Daemon.
1ldapsearch -x -h <ldap-host> -b "ou=People,dc=example,dc=com" uid=<user>
DN components:
cn : common nameou : organizational unito : organizationc : countrydc : domain component 1# LDIF = the commands between EOF
2# -W prompts for the LDAP admin password
3# -w passes the password (put it in a variable)
4# bind_dn : the DN that acts as the LDAP bind user
5
6export bind_dn="CN=directory manager,DC=example,DC=org"
7
8# Modify an entry
9ldapadd -W -D "$bind_dn" -h $ldap_server -p 389 <<EOT
10dn: cn=user,ou=wiki,dc=example,dc=com
11changetype: modify
12add: memberUid
13memberUid: $login
14EOT
15
16# Create a new entry
17ldapadd -w $LDAPpwd -D "$bind_dn" -h $ldap_server -p 389 <<EOT
18dn: uid=${login},ou=People,dc=example,dc=org
19uid: ${login}
20loginShell: /bin/bash
21uidNumber: ${uid}
22gidNumber: 47110
23homeDirectory: /home/${login}
24shadowLastChange: 0
25shadowMax: -1
26objectClass: account
27objectClass: posixaccount
28objectClass: shadowaccount
29objectClass: top
30gecos: ${gecos}
31cn: ${gecos}
32userPassword: {CRYPT}`perl -e 'print crypt("${login}", "${login}")'`
33EOT
| Unix proprietary | GNU / Linux | BSD / open source | Mainframe | Virtualisation |
|---|---|---|---|---|
| AIX | Debian | FreeBSD | MVS (IBM) | VMware / VirtualBox |
| HP-UX | Slackware | NetBSD | SCOS (Bull) | Cloud (IaaS) |
| SunOS (BSD fork) โ Solaris | SUSE | OpenBSD | OpenStack (IaaS/SaaS) | |
| IRIX (SGI) | Red Hat | FreeBSD โ macOS | ||
| Fedora | ||||
| openSUSE | ||||
| CentOS | ||||
| Ubuntu (Debian) | ||||
| Mint |
Historically, Unix split into two main branches: