*nix

From Things and Stuff Wiki
Revision as of 08:06, 24 July 2012 by Milk (talk | contribs) (→‎Commands)
Jump to navigation Jump to search


Mainly linux, some unix-like.

Guides

Dotfiles

System

Standards

  • POSIX, an acronym for "Portable Operating System Interface", is a family of standards specified by the IEEE for maintaining compatibility between operating systems. POSIX defines the application programming interface (API), along with command line shells and utility interfaces, for software compatibility with variants of Unix and other operating systems.
  • Linux Standard Base (LSB) is a joint project by several Linux distributions under the organizational structure of the Linux Foundation to standardize the software system structure, including the filesystem hierarchy, used with Linux operating system. The LSB is based on the POSIX specification, the Single UNIX Specification, and several other open standards, but extends them in certain areas.
  • freedesktop.org is open source / open discussion software projects working on interoperability and shared technology for X Window System desktops. The most famous X desktops are GNOME and KDE, but developers working on any Linux/UNIX GUI technology are welcome to participate. freedesktop.org is building a base platform for desktop software on Linux and UNIX. The elements of this platform have become the backend for higher-level application-visible APIs such as Qt, GTK+, XUL, VCL, WINE, GNOME, and KDE. The base platform is both software and specifications.

Boot

  • e4rat - reduce boot time (into X) by some 50% for ext4

Monitoring

  • lsof - "lists open files" (lots, given "everything" is a file)

iostat, vmstat, free

File structure

Mount

mount -o remount /
  remount partition after /etc/fstab change

Partitions

  • GNU Parted manipulates partition tables. This is useful for creating space for new operating systems, reorganizing disk usage, copying data on hard disks and disk imaging. The package contains a library, libparted, as well as well as a command-line frontend, parted, which can also be used in scripts.
  • http://gparted.sourceforge.net/
dd
  • dd - Copy a file, converting and formatting according to the options.
    • dd is a common Unix program whose primary purpose is the low-level copying and conversion of raw data.
dd if=/dev/sr0 of=myCD.iso bs=2048 conv=noerror,sync
  create an ISO disk image from a CD-ROM.
dd if=/dev/sda2 of=/dev/sdb2 bs=4096 conv=noerror
  Clone one partition to another
dd if=/dev/ad0 of=/dev/ad1 bs=1M conv=noerror
  Clone a hard disk "ad0" to "ad1".

dd if=/dev/zero bs=1024 count=1000000 of=file_1GB
dd if=file_1GB of=/dev/null bs=64k
  drive benchmark test and analyze the sequential read and write performance for 1024 byte blocks
du (disk usage)
du -sh
  size of a folder
du -S
  size of files in a folder

du -aB1m|awk '$1 >= 100'
  everything over 100Mb

cd / | sudo du -khs *
  show root folder size

sudo du -a --max-depth=1 /usr/lib | sort -n -r | head -n 20
  size of program folders /usr/lib

du -sk ./* | sort -nr | awk 'BEGIN{ pref[1]="K"; pref[2]="M"; pref[3]="G";} { total = total + $1; x = $1; y = 1; while( x > 1024 ) { x = (x + 1023)/1024; y++; } printf("%g%s\t%s\n",int(x*10)/10,pref[y],$2); } END { y = 1; while( total > 1024 ) { total = (total + 1023)/1024; y++; } printf("Total: %g%s\n",int(total*10)/10,pref[y]); }'
ncdu
  • ncdu - ncurses disk usage
ncdu / --exclude /home --exclude /media --exclude /run/media
  check everything apart from home and external drives

ncdu / --exclude /home --exclude /media --exclude /run/media
  check everything apart from external drives
df
  • df - report file system disk space usage
Baobab
Other

todo; source aliases.zsh

  • Filelight creates an interactive map of concentric, segmented rings that help visualise disk usage on your computer.

Files and directories

ls
  list in row
ls -l
  long list

ls *
  files in directory and immediate subdiretories

just names; 

ls -m1
  -m fill width with a comma separated list of entries ??
ls --format single-column
  column of names only
ls -l | grep - | awk '{print $9}'
  using awk to show the 9th word (name). strips colour.
ls -l | cut -f9 -s -d" "
  using cut to cut from the 9th word, using space as a delimiter. strips colour.
ls | cat
  neat

ls -a
  show hidden files
ls  -A
  show hidden files, exclude . and ..

stat .
  display file or file system status
stat -c "%n %a" * | column -t
  directory files + octal

mkdir directory
mkdir directory -p
  no error if existing, make parent directories as needed

ln -s {target-filename}
ln -s {target-filename} {symbolic-filename}



cd change/directory/path

[3] 

mv position1 ~/position2
  basic move

pax - read and write file archives and copy directory hierarchies 

find . -type f -print0 | xargs -0 stat -c "%y %s %n"

Copying files

cp - copy files and directories

scp -P 2264 foobar.txt your_username@remotehost.edu:/some/remote/directory
scp -rP 2264 folder your_username@remotehost.edu:/some/remote/directory

wget -O myzip.zip https://github.com/zeromq/jzmq/zipball/master
wget -k
  --convert-links

Viewing files

less is better than more 

cat filename
  output file to screen
cat -n filename
  output file to screen w/ line numbers
cat filename1 filename2
  output two files (concatinate)
cat filename1 > filename2
  overwrite filename2 with filename1
cat filename1 >> filename2
  append filename1 to filename2
cat filename{1,2} > filename2
  add filename1 and filename2 together into filename3

head filename
  top 10 lines of file
head -23 filename
  top 23 lines of file
tail filename
  bottom 10 lines of file
tail -23 filename
  bottom 23 lines of file

sed -n 20,30p filename
  print lines 20..30 of file [4]

Finding files

find /usr/share -name README
find ~/Journalism -name '*.txt'
find ~/Programming -path '*/src/*.c'

find ~/Images/Screenshots -size +500k -iname '*.jpg'
find ~/Journalism -name '*.txt' -exec cat {} ;
find ~/Journalism -name '*.txt' -print0 | xargs -0 cat   (faster than above)

find / -group [group]
find / -user [user]

http://arstechnica.com/information-technology/2011/07/ask-ars-how-to-use-the-find-command-in-a-pipeline/

  • sgrep - search a file for a structured pattern

Compression

tar
tar <operation> [options]
Operations:
  [-]A --catenate --concatenate
  [-]c --create
  [-]d --diff --compare
  [-]r --append
  [-]t --list
  [-]u --update
  [-]x --extract --get
  --delete

Common Options:
  -C, --directory DIR
  -f, --file F
  -j, --bzip2
  -p, --preserve-permissions
  -v, --verbose
  -z, --gzip
zip
  • 7-Zip is a file archiver with the highest compression ratio. The program supports 7z (that implements LZMA compression algorithm), ZIP, CAB, ARJ, GZIP, BZIP2, TAR, CPIO, RPM and DEB formats. Compression ratio in the new 7z format is 30-50% better than ratio in ZIP format.
    • p7zip is a port of 7za.exe for POSIX systems like Unix (Linux, Solaris, OpenBSD, FreeBSD, Cygwin, AIX, ...), MacOS X and also for BeOS and Amiga. 7za.exe is the command line version of 7-zip, see http://www.7-zip.org/. 7-Zip is a file archiver with highest compression ratio.
    • man z7 (p7zip)
    • p7zip-light in AUR
7z x filename
  extract archive with directories
pax
  • pax will read, write, and list the members of an archive file, and will copy directory hierarchies. pax operation is independent of the specific archive format, and supports a wide variety of different archive formats. A list of supported archive formats can be found under the description of the -x option. [5]
Generic function
# Extract Files
extract() {
 if [ -f $1 ] ; then
     case $1 in
         *.tar.bz2)   tar xvjf $1    ;;
         *.tar.gz)    tar xvzf $1    ;;
         *.tar.xz)    tar xvJf $1    ;;
         *.bz2)       bunzip2 $1     ;;
         *.rar)       unrar x $1     ;;
         *.gz)        gunzip $1      ;;
         *.tar)       tar xvf $1     ;;
         *.tbz2)      tar xvjf $1    ;;
         *.tgz)       tar xvzf $1    ;;
         *.zip)       unzip $1       ;;
         *.Z)         uncompress $1  ;;
         *.7z)        7z x $1        ;;
         *.xz)        unxz $1        ;;
         *.exe)       cabextract $1  ;;
         *)           echo "\`$1': unrecognized file compression" ;;
     esac
 else
     echo "\`$1' is not a valid file"
 fi
}

Terminals

Basics

Urxvt

tabbed is good, plus tmux.

Screen

config goes in ~/.screenrc 

escape ^Ww
  change escape key to w

Tmux

Better than screen if available.

  • tmux - terminal multiplexer

config goes in ~/.tmux.conf 

tmux lsc
  list clients

tmux detach-client -t /dev/pts/26
  remove other clients from session (if screensize is fucked)

Misc

Shell

Bash

man: echo

Basics
Options
More

Zsh

lshell

  • lshell is a shell coded in Python, that lets you restrict a user's environment to limited sets of commands, choose to enable/disable any command over SSH (e.g. SCP, SFTP, rsync, etc.), log user's commands, implement timing restriction, and more.

Inferno

  • Inferno is a distributed operating system, originally developed at Bell Labs, but now developed and maintained by Vita Nuova® as Free Software. Applications written in Inferno's concurrent programming language, Limbo, are compiled to its portable virtual machine code (Dis), to run anywhere on a network in the portable environment that Inferno provides. Unusually, that environment looks and acts like a complete operating system.

Fish

Commands

  • auditd - userspace component to the Linux Auditing System [8]
  • nice - run a program with modified scheduling priority
  • cut - remove sections from each line of files
  • shelr - console screencasting tool

~/.local/share/applications/mimeinfo.cache

Finding programs

whereis

apropos apropos vim
  search the whatis database for strings

Cron

Wildcards


MIME

Users

  • /etc/passwd - local user information
    • account:password:UID:GID:GECOS:directory:shell
  • /etc/shadow - restricted access encrypted password file
useradd username
  create user with defaults (no password)
useradd -D
  show defaults that will be used

useradd -m -g [initial_group] -G [additional_groups] -s [login_shell] [username]
  -m - add home dir
  -d - non-default home dir path
  -c comment
  -e 2006-06-30 - expires
useradd -m -g users -G audio,lp,optical,storage,video,wheel,games,power,scanner -s /bin/bash username

adduser
  interactive tool

userdel username
  remove user
userdel -r username
  remove user plus their home folder, mail spool

cat /etc/passwd | cut -d: -f 1,3,6 | grep "[5-9][0-9][0-9]"
  List users on a system [9]

Groups

groupadd [group]
  add group
gpasswd -a [user] [group]
  add user to group
gpasswd -d [user] [group]
  delete user from group

User must relogin for new group to take effect.

Permissions

ACLs

Partition must have acl set in /etc/fstab (and be remounted after). 

setfacl -m "u:username:permissions"
setfacl -m "u:uid:permissions"
  add permissions for user

setfacl -m "g:groupname:permissions"
setfacl -m "g:gid:permissions"
  add permissions for group

setfacl -m "u:user:rwx" file
  add read, write, execure perms for user for file
setfacl -Rm "u:user:rw" /dir
  add recursive read, write perms for user for dir
setfacl -Rdm "u:user:rw" /dir
  add recursive read, write perms for user for dir and make them default for future changes

Networking

sudo /etc/init.d/<service> restart
  ubuntu, restart a service
sudo /etc/rc.d/<service> stop | start | restart
  arch, service things

sudo sh -c "echo 'something' >> /etc/privilegedfile"
chown -R user:group . - change all and subitems [10]
  • route -n - display the host's networks and gateway
  • View The Daily Show, etc. in the UK, etc. Mofity HTTP headers; X-Forwarded-For "12.13.14.15" [12]

Media

Package management

Apt

dpkg --get-selections > installed-software
  create list of installed software

dpkg --set-selections < installed-software
dselect
  reinstall from list

Pacman

pacman -Syu
  upgrade system
pacman -Qo [file]
  check what package owns a file
pacman -Qqtd
  check whether there are any orphaned packages
pacman -Rsn packagename
  remove orphaned packages
  • cacheclean - Cleans up pacman packages. Users selects how many old versions to keep.
cacheclean {-p} {-v} <# of copies to keep>
# of copies to keep - (required) how many generations of each package.
-p - (optional) preview what would be deleted.
-v - (optional) show deleted packages.


Time

Convert unixtime into date; 

date -d @1337000000

ntpdate pool.ntp.org && hwclock --systohc && hwclock --adjust
  Synchronize both your system clock and hardware clock and calculate/adjust time drift.
  Do not run this command if you already have ntpd running! [14]

Logs

GUI

X

Window Managers

Openbox

Qt

qtconfig qt4

Distros

Arch

Debian

Ubuntu

CentOS

Red Hat Enterprise Linux (RHEL)

"Red Hat Enterprise Linux is available only through a paid subscription service that provides access to software updates and varying levels of technical support."

"Since Red Hat Enterprise Linux is based completely on free and open source software, Red Hat makes available the complete source code to its enterprise distribution through its FTP site to anybody who wants it. Accordingly, several groups have taken this source code and compiled their own versions of Red Hat Enterprise Linux, typically with the only changes being the removal of any references to Red Hat's trademarks and pointing the update systems to non-Red Hat servers. Groups which have undertaken this include CentOS (the 8th most popular Linux distribution as of November 2011), Oracle Linux, Scientific Linux, White Box Enterprise Linux, StartCom Enterprise Linux, Pie Box Enterprise Linux, X/OS, Lineox, and Bull's XBAS for high-performance computing. All provide a free mechanism for applying updates without paying a service fee to the distributor.""

If going down the based-on-RHEL route, consideration needs to be paid towards the update delay of the distros when a new RHEL release comes out.

Oracle Linux

Quicker to release, but the corperate atmosphere of Oracle has a baaad relationship with the open source world in general (Java lawsuits, etc). [15]

Scientific Linux (SL)

Other software

CUPS

Printing system.

Regex

Cisco

Media

To check;

Storage

VOIP

Retrieved from "https://wiki.thingsandstuff.org/index.php?title=*nix&oldid=2742"