*nix
Mainly linux, some unix-like.
- Vim - Text editor, etc.
- Git - Distributed version control
- IRC - Internet Relat Chat
- Bitlbee - IM
- Security
Guides
- The Art of Unix Programming Eric Steven Raymond
- The Linux Command Line - A Book By William E. Shotts, Jr.
- I introduced my 5-year-old and 2-year-old to startx and xmonad. They’re DELIGHTED! - June 20th, 2012
Dotfiles
System
Standards
- POSIX, an acronym for "Portable Operating System Interface", is a family of standards specified by the IEEE for maintaining compatibility between operating systems. POSIX defines the application programming interface (API), along with command line shells and utility interfaces, for software compatibility with variants of Unix and other operating systems.
- Linux Standard Base (LSB) is a joint project by several Linux distributions under the organizational structure of the Linux Foundation to standardize the software system structure, including the filesystem hierarchy, used with Linux operating system. The LSB is based on the POSIX specification, the Single UNIX Specification, and several other open standards, but extends them in certain areas.
- freedesktop.org is open source / open discussion software projects working on interoperability and shared technology for X Window System desktops. The most famous X desktops are GNOME and KDE, but developers working on any Linux/UNIX GUI technology are welcome to participate. freedesktop.org is building a base platform for desktop software on Linux and UNIX. The elements of this platform have become the backend for higher-level application-visible APIs such as Qt, GTK+, XUL, VCL, WINE, GNOME, and KDE. The base platform is both software and specifications.
Boot
- e4rat - reduce boot time (into X) by some 50% for ext4
Monitoring
- lsof - "lists open files" (lots, given "everything" is a file)
lsof -i :[port] what application using a specific port
iostat, vmstat, free
Terminals
Basics
- Wikipedia:Pseudo terminal, Wikipedia:Virtual console (computer user-interface), Wikipedia:Terminal emulator, Wikipedia:Control character, Wikipedia:Escape sequence, Wikipedia:Escape character, Wikipedia:C0 and C1 control codes, Wikipedia:Terminfo
- ASCII Characters for MPE Users (control char info)
- The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) by Joel Spolsky
- Codepoint, n. the position of a character in an encoding system.
- Charbase - A visual unicode database
Urxvt
tabbed is good, plus tmux.
Screen
config goes in ~/.screenrc
escape ^Ww change escape key to w
Tmux
Better than screen if available.
- tmux - terminal multiplexer
config goes in ~/.tmux.conf
- http://www.linuxcertif.com/man/1/tmux/
- http://blog.hawkhost.com/2010/07/02/tmux-%E2%80%93-the-terminal-multiplexer-part-2/
- http://www.linuxized.com/2010/05/switching-from-gnu-screen-to-tmux/comment-page-1/#comment-2148
tmux lsc list clients tmux detach-client -t /dev/pts/26 remove other clients from session (if screensize is fucked)
Misc
- ArchWiki: Improve Boot Performance#TTY terminal management
- StackOverflow: Can terminals detect <Shift-Enter> or <Control-Enter>?
- Home and End keys not working
- That 256 Color Thing, P.C. Shyamshankar
- Super User: TTY with 256 colors?
- The Text Pistols
- Wikipedia:Plan 9 from Bell Labs
- Gate One is an HTML5-powered terminal emulator and SSH client]
Shell
Bash
- Bash Reference Manual
- Builtin Commands
- Shell variables: Bourne, Bash
- Conditional Expressions
- Bash Guide for Beginners
- Advanced Bash-Scripting Guide by Mendel Cooper
- .bashrc, etc
- Getting Started with BASH
- The Command Line Crash Course (for cli newbs)
- Writing Robust Bash Shell Scripts
- http://blog.commandlinekungfu.com/p/index-of-tips-and-tricks.html Unix Command-Line Kung Fu] Tip index
- http://mywiki.wooledge.org/BashGuide/Practices
- http://www.aboutlinux.info/2005/10/10-seconds-guide-to-bash-shell.html
man: echo
Basics
- IBM - Linux tip: Bash parameters and parameter expansions
- Debugging a script Bash Hackers Wiki
Options
- http://rsalveti.wordpress.com/2007/04/03/bash-parsing-arguments-with-getopts/
- http://wiki.bash-hackers.org/howto/getopts_tutorial
- shFlags - getop wrapper for long flags with fallback for non gnu-getopt
More
Zsh
- Github: P.C. Shyamshankar's Zsh configs
- https://gist.github.com/914831
- https://github.com/olivierverdier/zsh-git-prompt or https://github.com/jcorbin/zsh-git
- http://quasimal.com/posts/2012-05-21-funsh.html - functional programming in zsh
lshell
- lshell is a shell coded in Python, that lets you restrict a user's environment to limited sets of commands, choose to enable/disable any command over SSH (e.g. SCP, SFTP, rsync, etc.), log user's commands, implement timing restriction, and more.
git-shell
- git-shell - Restricted login shell for Git-only SSH access
Inferno
- Inferno is a distributed operating system, originally developed at Bell Labs, but now developed and maintained by Vita Nuova® as Free Software. Applications written in Inferno's concurrent programming language, Limbo, are compiled to its portable virtual machine code (Dis), to run anywhere on a network in the portable environment that Inferno provides. Unusually, that environment looks and acts like a complete operating system.
Fish
File structure
See LSB, etc.
- Linux Directory Structure (File System Structure) Explained with Examples
- Point/Counterpoint - /opt vs. /usr/local - March 2010
- Understanding the bin, sbin, usr/bin , usr/sbin split [2] - Dec 9, 2010
- http://mailman.archlinux.org/pipermail/arch-dev-public/2012-March/022625.html [arch-dev-public] [RFC] merge /bin, /sbin, /lib into /usr/bin and /usr/lib] - Mar 2nd, 2012
Mount
mount -o remount / remount partition after /etc/fstab change
Partitions
- Arch Wiki: Partitioning
- GNU Parted manipulates partition tables. This is useful for creating space for new operating systems, reorganizing disk usage, copying data on hard disks and disk imaging. The package contains a library, libparted, as well as well as a command-line frontend, parted, which can also be used in scripts.
- http://gparted.sourceforge.net/
dd
- dd - Copy a file, converting and formatting according to the options.
- dd is a common Unix program whose primary purpose is the low-level copying and conversion of raw data.
dd if=/dev/sr0 of=myCD.iso bs=2048 conv=noerror,sync create an ISO disk image from a CD-ROM. dd if=/dev/sda2 of=/dev/sdb2 bs=4096 conv=noerror Clone one partition to another dd if=/dev/ad0 of=/dev/ad1 bs=1M conv=noerror Clone a hard disk "ad0" to "ad1".
dd if=/dev/zero bs=1024 count=1000000 of=file_1GB dd if=file_1GB of=/dev/null bs=64k drive benchmark test and analyze the sequential read and write performance for 1024 byte blocks
du (disk usage)
du -sh size of a folder du -S size of files in a folder du -aB1m|awk '$1 >= 100' everything over 100Mb
cd / | sudo du -khs * show root folder size sudo du -a --max-depth=1 /usr/lib | sort -n -r | head -n 20 size of program folders /usr/lib du -sk ./* | sort -nr | awk 'BEGIN{ pref[1]="K"; pref[2]="M"; pref[3]="G";} { total = total + $1; x = $1; y = 1; while( x > 1024 ) { x = (x + 1023)/1024; y++; } printf("%g%s\t%s\n",int(x*10)/10,pref[y],$2); } END { y = 1; while( total > 1024 ) { total = (total + 1023)/1024; y++; } printf("Total: %g%s\n",int(total*10)/10,pref[y]); }'
ncdu
- ncdu - ncurses disk usage
ncdu / --exclude /home --exclude /media --exclude /run/media check everything apart from home and external drives ncdu / --exclude /home --exclude /media --exclude /run/media check everything apart from external drives
df
- df - report file system disk space usage
Baobab
- Baobab - gnome app
Other
todo; source aliases.zsh
- Filelight creates an interactive map of concentric, segmented rings that help visualise disk usage on your computer.
Files and directories
ls list in row ls -l long list ls * files in directory and immediate subdiretories
just names;
ls -m1 -m fill width with a comma separated list of entries ?? ls --format single-column column of names only ls -l | grep - | awk '{print $9}' using awk to show the 9th word (name). strips colour. ls -l | cut -f9 -s -d" " using cut to cut from the 9th word, using space as a delimiter. strips colour. ls | cat neat ls -a show hidden files ls -A show hidden files, exclude . and ..
stat . display file or file system status stat -c "%n %a" * | column -t directory files + octal
mkdir directory mkdir directory -p no error if existing, make parent directories as needed
ln -s {target-filename} ln -s {target-filename} {symbolic-filename}
cd change/directory/path
mv position1 ~/position2 basic move
- http://superuser.com/questions/187866/unix-shell-scripting-how-to-recursively-move-files-up-one-directory
- http://serverfault.com/questions/122233/how-to-recursively-move-all-files-including-hidden-in-a-subfolder-into-a-paren
pax - read and write file archives and copy directory hierarchies
find . -type f -print0 | xargs -0 stat -c "%y %s %n"
Copying files
cp - copy files and directories
scp -P 2264 foobar.txt your_username@remotehost.edu:/some/remote/directory scp -rP 2264 folder your_username@remotehost.edu:/some/remote/directory
wget -O myzip.zip https://github.com/zeromq/jzmq/zipball/master wget -k --convert-links
Viewing files
less is better than more
cat filename output file to screen cat -n filename output file to screen w/ line numbers cat filename1 filename2 output two files (concatinate) cat filename1 > filename2 overwrite filename2 with filename1 cat filename1 >> filename2 append filename1 to filename2 cat filename{1,2} > filename2 add filename1 and filename2 together into filename3
head filename top 10 lines of file head -23 filename top 23 lines of file tail filename bottom 10 lines of file tail -23 filename bottom 23 lines of file
sed -n 20,30p filename print lines 20..30 of file [4]
Finding files
find /usr/share -name README find ~/Journalism -name '*.txt' find ~/Programming -path '*/src/*.c' find ~/Images/Screenshots -size +500k -iname '*.jpg' find ~/Journalism -name '*.txt' -exec cat {} ; find ~/Journalism -name '*.txt' -print0 | xargs -0 cat (faster than above) find / -group [group] find / -user [user]
- sgrep - search a file for a structured pattern
Compression
tar
tar <operation> [options] Operations: [-]A --catenate --concatenate [-]c --create [-]d --diff --compare [-]r --append [-]t --list [-]u --update [-]x --extract --get --delete Common Options: -C, --directory DIR -f, --file F -j, --bzip2 -p, --preserve-permissions -v, --verbose -z, --gzip
zip
- 7-Zip is a file archiver with the highest compression ratio. The program supports 7z (that implements LZMA compression algorithm), ZIP, CAB, ARJ, GZIP, BZIP2, TAR, CPIO, RPM and DEB formats. Compression ratio in the new 7z format is 30-50% better than ratio in ZIP format.
- p7zip is a port of 7za.exe for POSIX systems like Unix (Linux, Solaris, OpenBSD, FreeBSD, Cygwin, AIX, ...), MacOS X and also for BeOS and Amiga. 7za.exe is the command line version of 7-zip, see http://www.7-zip.org/. 7-Zip is a file archiver with highest compression ratio.
- man z7 (p7zip)
- p7zip-light in AUR
7z x filename extract archive with directories
pax
- pax will read, write, and list the members of an archive file, and will copy directory hierarchies. pax operation is independent of the specific archive format, and supports a wide variety of different archive formats. A list of supported archive formats can be found under the description of the -x option. [5]
Generic function
# Extract Files extract() { if [ -f $1 ] ; then case $1 in *.tar.bz2) tar xvjf $1 ;; *.tar.gz) tar xvzf $1 ;; *.tar.xz) tar xvJf $1 ;; *.bz2) bunzip2 $1 ;; *.rar) unrar x $1 ;; *.gz) gunzip $1 ;; *.tar) tar xvf $1 ;; *.tbz2) tar xvjf $1 ;; *.tgz) tar xvzf $1 ;; *.zip) unzip $1 ;; *.Z) uncompress $1 ;; *.7z) 7z x $1 ;; *.xz) unxz $1 ;; *.exe) cabextract $1 ;; *) echo "\`$1': unrecognized file compression" ;; esac else echo "\`$1' is not a valid file" fi }
Commands
Resources
- commandlinefu.com [6]
- LINFO: Index of Linux Commands
- Awesome but often unknown Linux commands and tools - August 10th, 2011
- LinuxCommand.org is a Linux education and advocacy site devoted to helping users of legacy operating systems migrate into the future.
- WikiBooks: Guide to Unix/Commands
- SHELLdorado - your UNIX shell scripting resource
Programs
- World's best introduction to sed [7]
- awk
- ack - ack is a perl tool like grep for programmers
- https://github.com/rupa/z / https://github.com/joelthelion/autojump - verrry handy for moving about
- nice - run a program with modified scheduling priority
- cut - remove sections from each line of files
- shelr - console screencasting tool
~/.local/share/applications/mimeinfo.cache
Finding programs
whereis
apropos apropos vim search the whatis database for strings
Cron
- Wikipedia: cron
- Arch Wiki: cron
- Gentoo Linux Cron Guide
crontab -l view crontabs crontab -e edit their crontabs crontab -r remove their crontabs crontab saved_crontab_filename overwrite their old crontab with saved crontab
There are several special predefined values which can be used to substitute the CRON expression.
Entry Description Equivalent To @yearly (or @annually) Run once a year, midnight, Jan. 1st 0 0 1 1 * @monthly Run once a month, midnight, first of month 0 0 1 * * @weekly Run once a week, midnight on Sunday 0 0 * * 0 @daily Run once a day, midnight 0 0 * * * @hourly Run once an hour, beginning of hour 0 * * * * @reboot Run at startup @reboot
* * * * * command to be executed ┬ ┬ ┬ ┬ ┬ │ │ │ │ │ │ │ │ │ │ │ │ │ │ └───── day of week (0 - 6) (0 is Sunday, or use names) │ │ │ └────────── month (1 - 12) │ │ └─────────────── day of month (1 - 31) │ └──────────────────── hour (0 - 23) └───────────────────────── min (0 - 59)
- /etc/cron.allow - If this file exists, then you must be listed therein (your username must be listed) in order to be allowed to use cron jobs.
- /etc/cron.deny - If the cron.allow file does not exist but the /etc/cron.deny file does exist, then you must not be listed in the /etc/cron.deny file in order to use cron jobs.
Wildcards
- tuxfiles.org: wildcards
MIME
Users
- /etc/passwd - local user information
- account:password:UID:GID:GECOS:directory:shell
- /etc/shadow - restricted access encrypted password file
useradd username create user with defaults (no password) useradd -D show defaults that will be used useradd -m -g [initial_group] -G [additional_groups] -s [login_shell] [username] -m - add home dir -d - non-default home dir path -c comment -e 2006-06-30 - expires useradd -m -g users -G audio,lp,optical,storage,video,wheel,games,power,scanner -s /bin/bash username
adduser interactive tool
userdel username remove user userdel -r username remove user plus their home folder, mail spool
cat /etc/passwd | cut -d: -f 1,3,6 | grep "[5-9][0-9][0-9]" List users on a system [9]
Groups
groupadd [group] add group gpasswd -a [user] [group] add user to group gpasswd -d [user] [group] delete user from group
User must relogin for new group to take effect.
Permissions
- Chmod, Umask, Stat, Fileperms, and File Permissions - 0000 to 0777 list, etc.
ACLs
Partition must have acl set in /etc/fstab (and be remounted after).
setfacl -m "u:username:permissions" setfacl -m "u:uid:permissions" add permissions for user setfacl -m "g:groupname:permissions" setfacl -m "g:gid:permissions" add permissions for group setfacl -m "u:user:rwx" file add read, write, execure perms for user for file setfacl -Rm "u:user:rw" /dir add recursive read, write perms for user for dir setfacl -Rdm "u:user:rw" /dir add recursive read, write perms for user for dir and make them default for future changes
Networking
sudo /etc/init.d/<service> restart ubuntu, restart a service sudo /etc/rc.d/<service> stop | start | restart arch, service things
sudo sh -c "echo 'something' >> /etc/privilegedfile" chown -R user:group . - change all and subitems [10]
- Arch Wiki: Configuring Network
- Linux DNS Lookup Command - host & dig
- route -n - display the host's networks and gateway
- /etc/hosts
- /etc/network/interfaces - ubuntu network interface settings
- /etc/resolv.conf - dns settings
- /etc/host.conf - dns resolve order
- /etc/dhcp3/dhclient.conf - overrides ubuntu server dns settings. change prepend option.
- /etc/dnsmasq.conf - dnsmasq settings
- dnsmasq configuration
- Local DNS cache in Linux using dnsmasq
- View The Daily Show, etc. in the UK, etc. Mofity HTTP headers; X-Forwarded-For "12.13.14.15" [12]
curl
- curl is a tool to transfer data from or to a server, using one of the supported protocols (DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and TFTP). The command is designed to work without user interaction.
curl http://site.{one,two,three}.com
curl ftp://ftp.numericals.com/file[1-100].txt ftp://ftp.numericals.com/file[001-100].txt (with leading zeros) ftp://ftp.letters.com/file[a-z].txt sequences of alphanumeric series by using []
curl http://any.org/archive[1996-1999]/vol[1-4]/part{a,b,c}.html Nested sequences are not supported, but you can use several ones next to each other:
curl http://www.numericals.com/file[1-100:10].txt http://www.letters.com/file[a-z:2].txt multiple urls + specify a step counter for the ranges to get every Nth number or letter:
nmap
nmap -sT -sU -O -p 1-65535 localhost full port scan
VOIP
Media
Package management
Apt
dpkg --get-selections > installed-software create list of installed software dpkg --set-selections < installed-software dselect reinstall from list
Pacman
pacman -Syu upgrade system pacman -Qo [file] check what package owns a file pacman -Qqtd check whether there are any orphaned packages pacman -Rsn packagename remove orphaned packages
- cacheclean - Cleans up pacman packages. Users selects how many old versions to keep.
cacheclean {-p} {-v} <# of copies to keep> # of copies to keep - (required) how many generations of each package. -p - (optional) preview what would be deleted. -v - (optional) show deleted packages.
Time
Convert unixtime into date;
date -d @1337000000
- Linux, Clocks, and Time
- tzconfig / dpkg-reconfigure tzdata
ntpdate pool.ntp.org && hwclock --systohc && hwclock --adjust Synchronize both your system clock and hardware clock and calculate/adjust time drift. Do not run this command if you already have ntpd running! [14]
Powersaving
GUI
- http://www.tomshardware.com/reviews/image-software-linux-gimp,2801-3.html
- http://linuxcrunch.com/content/gwenview-best-image-viewer
X
Window Managers
Openbox
Qt
qtconfig qt4
Other software
- http://kmandla.wordpress.com/software/
- http://blog.chavezgu.com/2012/03/07/the-command-line-challenge/
- shortcutworld.com - keyboard shortcuts wiki
- https://www.shortcutfoo.com/
CUPS
Printing system.