Linux

The Basics NFS vs iscsi NFS can handle simultaniously writing from several clients. NFS is a filesystem , iscsi is a block storage. iscsi performance are same with NFS. iscsi will appear as disk to the OS, not the case for NFS. Concurrent access to a block device like iSCSI is not possible with standard file systems. You’ll need a shared disk filesystem (like GFS or OCSFS) to allow this, but in most cases the easiest solution would be to just use a network share (via SMB/CIFS or NFS) if this is sufficient for your application.

Ressources 1# in crontab or tmux session - take every hour a track of the memory usage 2for i in {1..24} ; do echo -n "===================== " ; date ; free -m ; top -b -n1 | head -n 15 ; sleep 3600; done >> /var/log/SYSADM/memory.log &

Compare files 1diff <file1> <file2> # -w to ignore whitespace. 2colordiff <file1> <file2> # colourised diff. 3wdiff <file1> <file2> # word diff: [− −] replaced word, {+ +} added word. 4vimdiff <file1> <file2> # open both files in vim (blue = entirely different lines, red = partially different). 5fgrep -f <list> <file> # compare two lists (e.g. of hosts). Compare jar files 1diff -W200 -y <(unzip -vqq file1.jar | awk '{ if ($1 > 0) {printf("%s\t%s\n", $1, $8)}}' | sort -k2) <(unzip -vqq file2.jar | awk '{ if ($1 > 0) {printf("%s\t%s\n", $1, $8)}}' | sort -k2)

Find a process blocking a file with fuser: 1fuser -m </dir or /files> # Find process blocking/using this directory or files. 2fuser -cu </dir or /files> # Same as above but add the user 3fuser -kcu </dir or /files> # Kill process 4fuser -v -k -HUP -i ./ # Send HUP signal to process 5 6# Output will send you <PID + letter>, here is the meaning: 7# c current directory. 8# e executable being run. 9# f open file. (omitted in default display mode). 10# F open file for writing. (omitted in default display mode). 11# r root directory. 12# m mmap'ed file or shared library. with lsof ( = list open file): 1lsof +D /var/log # Find all files blocked with the process and user. 2lsof -a +L1 <mountpoint> # Process blocking a FS. 3lsof -c ssh -c init # Find files open by thoses processes. 4lsof -p 1753 # Find files open by PID process. 5lsof -u root # Find files open by user. 6lsof -u ^user # Find files open by user except this one. 7kill -9 `lsof -t -u toto` # kill user's processes. (option -t output only PID). MacGyver method: 1#When you have no fuser or lsof: 2find /proc/*/fd -type f -links 0 -exec ls -lrt {} \; AIX specifics (fuser) 1fuser -d /tmp # see the processes using the /tmp directory (AIX) 2fuser -c /your_FS # all processes with an open file in the filesystem (AIX) 3fuser -cu /dev/vg01/lvol5 # also search with a filesystem or an LV -c == -m ; -u also shows the process user. to kill the processes: fuser -kcu. File deleted but space still held For detecting deleted-but-still-open files (lsof +L1) and freeing the held space, see the Disk Cleanup page.

Basic Troubleshooting Checks interfaces 1nmcli con show 2NAME UUID TYPE DEVICE 3ens192 4d0087a0-740a-4356-8d9e-f58b63fd180c ethernet ens192 4ens224 3dcb022b-62a2-4632-8b69-ab68e1901e3b ethernet ens224 5 6nmcli dev status 7DEVICE TYPE STATE CONNECTION 8ens192 ethernet connected ens192 9ens224 ethernet connected ens224 10ens256 ethernet connected ens256 11lo loopback unmanaged -- 12 13# Get interfaces details : 14nmcli connection show ens192 15nmcli -p con show ens192 16 17# Get DNS settings in interface 18UUID=$(nmcli --get-values connection.uuid c show "cloud-init eth0") 19nmcli --get-values ipv4.dns c show $UUID Changing Interface name 1nmcli connection add type ethernet mac "00:50:56:80:11:ff" ifname "ens224" 2nmcli connection add type ethernet mac "00:50:56:80:8a:0b" ifname "ens256" Create a custom config 1nmcli con load /etc/sysconfig/network-scripts/ifcfg-ens224 2nmcli con up ens192 Adding a Virtual IP 1nmcli con mod enp1s0 +ipv4.addresses "192.168.122.11/24" 2ip addr del 10.10.10.36/24 dev ens160 3 4nmcli con reload # before to reapply 5nmcli device reapply ens224 6systemctl status network.service 7systemctl restart network.service Add a DNS entry 1UUID=$(nmcli --get-values connection.uuid c show "cloud-init eth0") 2DNS_LIST=$(nmcli --get-values ipv4.dns c show $UUID) 3nmcli conn modify "$UUID" ipv4.dns "${DNS_LIST} ${DNS_IP}" 4 5# /etc/resolved is managed by systemd-resolved 6sudo systemctl restart systemd-resolved

Server Side First Install samba and samba-client (for debug + test) /etc/samba/smb.conf 1[home] 2Workgroup=WORKGROUP (le grp par defaul sur windows) 3Hosts allow = ... 4[shared] 5browseable = yes 6path = /shared 7valid users = user01, @un_group_au_choix 8writable = yes 9passdb backend = tdbsam #passwords are stored in the /var/lib/samba/private/passdb.tdb file. Test samba config testparm /usr/bin/testparm -s /etc/samba/smb.conf smbclient -L \192.168.56.102 -U test : list all samba shares available smbclient //192.168.56.102/sharedrepo -U test : connect to the share pdbedit -L : list user smb (better than smbclient)

SSHFS SSHFS mounts a remote filesystem on your local filesystem through an SSH connection, all with user rights. The advantage is being able to manipulate remote data with any file manager (Nautilus, Konqueror, ROX, or even the command line). - Prerequisites: administrator rights, ethernet connection, installation of FUSE and the SSHFS package. - SSHFS users must belong to the `fuse` group. Note: FUSE allows a user to mount a filesystem themselves. Normally, mounting a filesystem requires being an administrator, or having it pre-approved in /etc/fstab with hard-coded information.

Investigate a user 1last # the last user connections to a server (based on /var/log/wtmp or btmp). 2ac -d # statistics of my connection time per day. 3ac -p <user> # the connection time of all users (or of a specific user). 4finger # who is connected (-l to also see mails and plans of all users). 5w # who is connected, doing what, and how much CPU they use. 6who # who is connected (-u for more info: PID, etc.). 7who am i # with which login I am connected. 8id -a # all info about the user I'm connected as (more precise than "who am i"). 9logname # the login name of the current account. Reboots & uptime 1last reboot # see all the reboots that took place. 2uptime # see how long the server has been up + the load average. 3lslogins -L # also shows whether a user shutdown/rebooted the machine.

Where the system logs live On a Unix machine, the system logs are in /var/log/messages (or /var/adm/messages on Solaris). This is where you find the errors, with log rotation. Default syslog output Linux Solaris HP-UX AIX BSD location /var/log/messages, /var/log/secure, /var/log/boot.log /var/adm/messages /var/adm/syslog/mail.log, /var/adm/syslog/syslog.log /tmp or none /var/log/syslog System accounting (login & process) Type Linux Solaris HP-UX AIX current logins /var/run/utmp /var/adm/utmpx /var/adm/utmp /etc/utmp login history /var/log/wtmp /var/adm/wtmpx /var/adm/wtmp /var/adm/wtmp process accounting /var/log/pacct /var/adm/pacct /var/adm/pacct /var/adm/pacct Login errors Linux Solaris HP-UX AIX failed logins /var/log/btmp, /var/log/messages /var/adm/loginlog, /var/adm/sulog /var/adm/sulog /etc/security/failedlogin Investigate the logs 1# today's logs 2grep "$(date '+%b %d')" /var/log/messages 3 4# disk errors (nawk: print the last field of the "Error Block" lines) 5nawk '/Error Block/{print $NF}' /var/adm/messages* | sort | uniq 6 7# find the IPs in a log, sort them and remove the duplicates 8cat /var/log/maillog | grep -Eo '([0-9]{1,3}\.){3}[0-9]{1,3}' | sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4 | uniq Network investigation 1# ping a list of servers 2for ip in $(awk '/192.168.45/ {print $1}' /etc/hosts); do ping -c 1 $ip; done Investigate on several servers 1for vm in vm{1..27}; do ssh -q $vm "hostname; free; sar -r 3 3"; done SSH authentication logs /var/log/auth.log — SSH connection logs. Check that there are not too many failed connections (a sign of an intrusion attempt).

Find files quickly 1locate <pattern> # find a directory or file quickly (uses an index); a brand-new file won't be found. 2updatedb # update the locate index. Open a file 1view <file> # opens a read-only vi view (preferred if you just want to search/view). 2gzcat / zcat <file.gz> # read a gzipped file. Info on a file or directory 1stat </my/file> # all info about a file (inode, creation, modification, access dates, etc.). 2stat -f <FS> # info about a filesystem. 3stat -c%s $LOGFILE # [scripting] get a precise value (size, modification date, etc.). grep 1grep -w 'xyz' # match the whole word. 2grep -x 'Hello, world!' # the whole line must match. 3grep -c <pattern> # count the matching lines. 4grep -l "ERROR:" *.log # search all .log files, list the files that match. 5grep -L <pattern> # inverse: list the files that do NOT match. 6grep -f <patternfile> <file> # apply the patterns read from patternfile. 7grep -i <pattern> # ignore case. 8grep -v <pattern> # return the lines that do NOT match. 9grep -m x <pattern> # stop after x matching lines. 10grep -n <pattern> # show the line number. 11grep -q <pattern> # quiet: exit 0 if found, 1 (or 2) otherwise (for scripting). 12grep -s <pattern> # suppress permission/inexistent-file error messages. 13grep -H <pattern> # show the filename next to each matching line. 14grep -h <pattern> # do not show the filename (default behaviour). 15grep -A x <pattern> # also show x lines After. 16grep -B x <pattern> # also show x lines Before. 17grep -C x <pattern> # show x lines of context (A + B). 18grep -a <pattern> <binary> # search a binary file as if it were text. 1egrep = grep -E # for complex regular expressions. 1# extract the 3rd field, then cut: 2cat file | grep /u01/grid/19c | awk '{print $3}' | cut -f2 -d'"' 3# is equivalent to: 4cat file | grep -o /u01/grid/19c

Configurations /etc/crontab : the daemon’s configuration file, defining the default behaviour of crond (SHELL, MAILTO, etc.). /etc/cron.d/... : system crontab (used by admins). /var/spool/cron/root : crontab per user. “Day of month” and “Day of week” are combined with a logical OR, so if both are set, the job runs on that day of the month and on that day of the week. 1MAILTO="admin@example.com" 2* * * * * root /usr/local/sbin/mycommand.sh > /dev/null 2>&1 Anacron /etc/anacrontab : file that runs, via the run-parts command, /etc/cron.daily, /etc/cron.weekly, /etc/cron.monthly. /etc/cron.d/0hourly : exception, runs /etc/cron.hourly via run-parts, checking the last run of the task in /var/spool/anacron/.... Special cases Exactly the last day of each month:

1logrotate -d /etc/logrotate.d/app # test a new configuration. 2reading config info for /data/log/app 3Handling 1 logs 4rotating pattern: /data/log/app after 1 days (10 rotations) 5empty log files are rotated, old logs are removed Options Usage: logrotate [OPTION...] <configfile> -d, --debug Don't do anything, just test (implies -v) -f, --force Force file rotation -m, --mail=command Command to send mail (instead of `/bin/mail') -s, --state=statefile Path of state file -v, --verbose Display messages during rotation