Memo

/etc/pam.d In /etc/pam.d, there is one PAM file per service. Syntax: module_type control_flag path_to_module_agent Module types auth — authentication. account — account-based restrictions (validity, time of day, etc.). session — things that run at login/logout. password — password updates. Control flags required — success needed; a failure is reported but only after the rest of the stack has run. requisite — like required, but a failure returns immediately without running the rest of the stack. sufficient — if this module succeeds, it is the last module tested in the stack. optional — its result is only taken into account if no other module succeeded or failed. [value=action value=action2 ...] — advanced control: map a module result to a specific action. Sample:

/etc/sudoers The /etc/sudoers file contains the set of UNIX operating-system privileges that the local administrator has granted to UNIX users. In no case should this file be edited directly with vi; it must be edited with visudo. 1sudo : execute a command as root. 2sudo su : become root and stay root. 3sudoers : file listing the commands allowed for certain users as the superuser (or another user). 4visudo -cs : strict syntax check of the sudoers file. sudo -i is equivalent to su - in terms of rights. with sudo -i, the user password is asked. with su -, the root password is asked. Verification 1sudo -l -U <user> Rules 1# "user" runs "sudo -u target All_the_commands" 2user server=(target) NOPASSWD: ALL With aliases 1Host_Alias LOAD_BALANCERS = server1,server2 2 3Cmnd_Alias SET_VIP = \ 4/sbin/ip addr add 192.168.10.12/20 broadcast 192.168.15.255 dev eth0 label eth0\:0, \ 5/sbin/ip addr del 192.168.10.12/20 dev eth0, \ 6/sbin/arping -U -c 1 -I eth0 192.168.10.12 7 8loaduser LOAD_BALANCERS=(root) NOPASSWD: SET_VIP 9syncuser LOAD_BALANCERS=(root) NOPASSWD: SET_VIP

Schedule a task (at / batch) 1at : schedule a task to run at a later time (/!\ it executes what you give it on stdin). 2 ex : at 18:22 < "date; ps -ef | wc -l" 3 or : at now + 5 hours then type your commands then ctrl + d 4at -q a 16:05 tomorrow : -q defines the queue (a-z), i.e. the priority. 5at -c <job_number> : see the context and the commands of the task. 6atq : list the pending jobs (= at -l). 7atrm : delete a job. 8 9batch : schedule a task when the load average is below a threshold. Run jobs in the background 1jobs -l : list the running tasks. 1# Nohup in series 2for i in {1..6}; do echo "nohup sh -c \"shred -vfz -n 3 /dev/cciss/c0d${i} > nohup${i}.out 2>&1 \" &" ; done | bash Three points to remember:

Source : http://www.glennklockwood.com/hpc-howtos/process-affinity.html Taskset 1sudo apt-get install util-linux # or: yum install util-linux 2taskset -c 1 script.sh # run script.sh on CPU number 1 (-c: CPU, -p: PID) 3taskset -c 1,2,3 script.sh # give it several CPUs. Numactl 1numactl --cpunodebind=0 simulation.x 2numactl --cpunodebind=0 --membind=0 simulation.x 3numactl -C 0 -N 0 simulation.x 4numactl -C +0,1,2,3 simulation.x # similar to taskset 5numactl -H # see which memory corresponds to a CPU Note: with numactl, unlike taskset, you cannot change the CPU affinity of a process on the fly.

Strace - trace system calls 1strace -tt -p 24503 2 3strace -o strace01.out -e open -f bash --login -i # see the files opened during a bash connection. 4 # -o redirects the output / -e filters the system calls / -f follows forks (child processes) 5 6strace -f <binary_script> 2> trace.log : stdout -> the binary command result, stderr -> the binary's system calls. Ltrace - trace library calls ltrace traces shared-library calls (like strace, but at the library-call level).

pidstat reports the CPU, memory, I/O and context-switch activity of processes. Report the process context-switching activity 1# pidstat -w -p 3446 2 5 2Linux 3.10.0-123.13.2.el7.x86_64 (localhost.localdomain) 12/26/2014 3_x86_64_ (1 CPU) 407:23:38 AM UID PID cswch/s nvcswch/s Command 507:23:40 AM 0 3446 0.50 0.00 sshd 607:23:42 AM 0 3446 0.50 0.00 sshd 707:23:44 AM 0 3446 0.50 0.00 sshd 807:23:46 AM 0 3446 0.50 0.00 sshd 907:23:48 AM 0 3446 0.50 0.00 sshd 10Average: 0 3446 0.50 0.00 sshd cswch/s : number of voluntary context switches the task made per second. (A voluntary context switch occurs when a task blocks because it requires a resource that is unavailable.) nvcswch/s : number of non-voluntary context switches the task made per second. (An involuntary context switch takes place when a task executes for the duration of its time slice and is then forced to relinquish the processor.) Page faults and memory 1pidstat -r -p <PID> 3600 72 # every hour, 72 times - practical for long-term monitoring. 2 3pidstat -r -p <PID> 50 12 407:26:44 PM PID minflt/s majflt/s VSZ RSS %MEM Command 507:27:34 PM 13775 1.64 0.00 34957320 18183312 55.30 java minflt/s : number of minor faults the task has made per second — those which did not require loading a memory page from disk. majflt/s : number of major faults the task has made per second — those which required loading a memory page from disk. VSZ : Virtual Size — the virtual memory usage of the entire task in kilobytes. RSS : Resident Set Size — the non-swapped physical memory used by the task in kilobytes. Disk I/O 1pidstat -d -p <PID> 50 12 pidstat -d reports I/O statistics (kernels 2.6.20 and later only). The following values are displayed:

Find a process 1ps -fp <pid> # find a process by its PID. 2pidof httpd # find the PIDs of httpd. 3pidstat -lp <pid> # process name with all its complete arguments. 4 # for a tomcat or weblogic process, you can split the arguments with: sed 's/ -/\n -/g' 5pidstat -C "mysql" # find a process by its name (gives the PID and CPU load). The process tree 1pstree -pu # the process tree with PID and user (if pstree is not installed, use the alternatives below). 1ps -ejH 2 PID PGID SID TTY TIME CMD 3 1 1 1 ? 00:00:37 init 411016 11009 11009 ? 00:00:00 sshd 511017 11017 11017 pts/12 00:00:00 bash 611125 11125 11017 pts/12 00:00:00 telnet 1ps axjf 2 PPID PID PGID SID TTY TPGID STAT UID TIME COMMAND 3 0 1 1 1 ? -1 Ss 0 0:37 init [5] 4 8617 10610 10610 10610 ? -1 Ss 0 0:00 \_ sshd: support [priv] 510610 10710 10610 10610 ? -1 S 5027 0:00 \_ sshd: support@notty 610710 10711 10711 10711 ? -1 Ss 5027 0:00 \_ sshd: support@internal-sftp-server 1ps faux 2USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND 3root 1 0.0 0.0 10372 696 ? Ss Aug09 0:37 init [5] 4user1 4168 0.0 0.0 8728 968 ? Ss Aug24 0:00 | \_ /bin/bash -c perl /data/supports/scripts/SRAM_asr5k.pl &>/dev/null 5user1 4174 0.0 0.0 34068 5044 ? S Aug24 0:00 | \_ perl /data/supports/SRAM_asr5k.pl 6user1 4188 0.0 0.0 8728 984 ? S Aug24 0:00 | \_ sh -c grep -c SRAM /data/syslogCOLLECT/LTE_*/*/*.20160824.log /proc The /proc filesystem exposes per-process information:

Out-of-band management refers to accessing and controlling a server (console, power on/off, BIOS, remote media) through a dedicated management channel that is separate from the normal data path. “In-band” means going through the OS and its network interface; “out-of-band” uses an independent controller (a BMC — Baseboard Management Controller) with its own network port, so it still works even when the OS is down, the machine is hung, or the network stack is broken. RSA is one vendor/technology family of out-of-band access.

Verify an ISO image To verify that an ISO image is good: compute its checksum with a sha1 or sha256 key, then compare it with the key published on the official website. 1sha256sum image.iso 2sha1sum image.iso

KickStart Technology that allows deploying servers with a predefined configuration (Red Hat’s equivalent of Solaris JumpStart). Default volume manager : LVM. See the LVM page for details.

The Unix variants Unix proprietary GNU / Linux BSD / open source Mainframe Virtualisation AIX Debian FreeBSD MVS (IBM) VMware / VirtualBox HP-UX Slackware NetBSD SCOS (Bull) Cloud (IaaS) SunOS (BSD fork) → Solaris SUSE OpenBSD OpenStack (IaaS/SaaS) IRIX (SGI) Red Hat FreeBSD → macOS Fedora openSUSE CentOS Ubuntu (Debian) Mint The Unix families Early SunOS (1–4) was BSD-derived; from Solaris 2 / SunOS 5 onward it is System V (SVR4) based — hence Solaris is listed under System V while the variants table notes its BSD fork. Historically, Unix split into two main branches:

How to approach performance monitoring and tuning in Linux, and the various subsystems (and performance metrics) that need to be monitored. On a very high level, the following four subsystems need to be monitored: CPU Memory I/O Network 1. CPU Four critical performance metrics for the CPU: context switch, run queue, CPU utilization, and load average. Context Switch When the CPU switches from one process (or thread) to another, it is called a context switch. When a process switch happens, the kernel stores the current state of the CPU (of a process or thread) in memory. The kernel also retrieves the previously stored state (of a process or thread) from memory and puts it in the CPU. Context switching is essential for multitasking of the CPU. However, a higher level of context switching can cause performance issues. Run Queue The run queue indicates the total number of active processes in the current queue for the CPU. When the CPU is ready to execute a process, it picks it up from the run queue based on the priority of the process. Note that processes that are in a sleep state, or I/O wait state, are not in the run queue. A higher number of processes in the run queue can therefore cause performance issues. CPU Utilization Indicates how much of the CPU is currently being used. 100% CPU utilization means the system is fully loaded. Load Average Indicates the average CPU load over a specific time period. On Linux, load average is displayed for the last 1 minute, 5 minutes, and 15 minutes. This helps to see whether the overall load on the system is going up or down. For example, a load average of 0.75 1.70 2.10 indicates that the load is coming down (0.75 = last 1 minute, 1.70 = last 5 minutes, 2.10 = last 15 minutes). Note that this load average is calculated by combining both the total number of processes in the queue, and the total number of processes in the uninterruptible task status. 2. Network A good understanding of TCP/IP concepts is helpful when analyzing any network issue. For network interfaces, monitor the total number of packets (and bytes) received/sent through the interface, the number of packets dropped, etc. 3. I/O I/O wait is the amount of time the CPU is waiting for I/O. Consistent high I/O wait on the system indicates a problem in the disk subsystem. Monitor reads/second and writes/second. These are measured in blocks, i.e. the number of blocks read/written per second. They are also referred to as bi and bo (block in and block out). tps indicates total transactions per second, which is the sum of rtps (read transactions per second) and wtps (write transactions per second). 4. Memory RAM is the physical memory. If you have 4 GB of RAM installed, you have 4 GB of physical memory. Virtual memory = swap space available on disk + physical memory. The virtual memory contains both user space and kernel space. Using a 32-bit or a 64-bit system makes a big difference in determining how much memory a process can use: On a 32-bit system a process can only access a maximum of 4 GB of virtual memory. On a 64-bit system there is no such limitation. Unused RAM is used by the kernel as filesystem cache. Linux swaps when it needs more memory than the physical memory. When it swaps, it writes the least-used memory pages from the physical memory to the swap space on the disk. Lots of swapping can cause performance issues: the disk is much slower than the physical memory, and it takes time to swap the memory pages from RAM to disk. The subsystems are interrelated All four subsystems are interrelated. Just because you see a high reads/second, writes/second, or I/O wait, it does not mean the issue is with the I/O subsystem. It also depends on what the application is doing. In most cases, the performance issue is caused by the application running on the Linux system.