*nix

From Things and Stuff Wiki
Revision as of 07:03, 27 November 2012 by Milk (talk | contribs) (→‎Finding files)
Jump to navigation Jump to search


Mainly Linux.

  • Vim - Text editor, etc.
  • Git - Distributed version control
  • IRC - Internet Relat Chat
  • Bitlbee - Instant Messaging
  • Security

Guides

Articles

Dotfiles

System

Standards

  • POSIX, an acronym for "Portable Operating System Interface", is a family of standards specified by the IEEE for maintaining compatibility between operating systems. POSIX defines the application programming interface (API), along with command line shells and utility interfaces, for software compatibility with variants of Unix and other operating systems.
  • The Single UNIX Specification (SUS) is the collective name of a family of standards for computer operating systems to qualify for the name "Unix". The SUS is developed and maintained by the Austin Group, based on earlier work by the IEEE and The Open Group.
  • Linux Standard Base (LSB) is a joint project by several Linux distributions under the organizational structure of the Linux Foundation to standardize the software system structure, including the filesystem hierarchy, used with Linux operating system. The LSB is based on the POSIX specification, the Single UNIX Specification, and several other open standards, but extends them in certain areas.
  • freedesktop.org is open source / open discussion software projects working on interoperability and shared technology for X Window System desktops. The most famous X desktops are GNOME and KDE, but developers working on any Linux/UNIX GUI technology are welcome to participate. freedesktop.org is building a base platform for desktop software on Linux and UNIX. The elements of this platform have become the backend for higher-level application-visible APIs such as Qt, GTK+, XUL, VCL, WINE, GNOME, and KDE. The base platform is both software and specifications.

Boot

  • e4rat - reduce boot time (into X) by some 50% for ext4

"In order to boot Arch Linux, a Linux-capable boot loader such as GRUB(2), Syslinux, LILO or GRUB Legacy must be installed to the Master Boot Record or the GUID Partition Table. The boot loader is responsible for loading the kernel and initial ramdisk before initiating the boot process."

  • init (short for initialization) is a daemon process that is the direct or indirect ancestor of all other processes. It automatically adopts all orphaned processes. Init is the first process started during booting, and is typically assigned PID number 1. It is started by the kernel using a hard-coded filename, and if the kernel is unable to start it, a kernel panic will result. Init continues running until the system is shut down. The design of init has diverged in Unix systems such as System III and System V, from the functionality provided by the init in Research Unix and its BSD derivatives. The usage on most Linux distributions is compatible with System V, but some distributions, such as Slackware, use a BSD-style and others, such as Gentoo, have their own customized version. Several replacement init implementations have been written which attempt to address design limitations in the standard versions. These include systemd and Upstart, the latter being used by Ubuntu and some other Linux distributions.
  • Udev loads kernel modules by utilizing coding parallelism to provide a potential performance advantage versus loading these modules serially. The modules are therefore loaded asynchronously. The inherent disadvantage of this method is that udev does not always load modules in the same order on each boot. If the machine has multiple block devices, this may manifest itself in the form of device nodes changing designations randomly. For example, if the machine has two hard drives, /dev/sda may randomly become /dev/sdb.

Articles

UNIX

Linux

Modules

  • Kernel modules are pieces of code that can be loaded and unloaded into the kernel upon demand. They extend the functionality of the kernel without the need to reboot the system.
  • modprobe is a Linux program originally written by Rusty Russell and used to add a loadable kernel module (LKM) to the Linux kernel or to remove an LKM from the kernel. It is commonly used indirectly: udev relies upon modprobe to load drivers for automatically detected hardware.
  • Dynamic Kernel Module Support (DKMS) is a framework used to generate Linux kernel modules whose sources do not generally reside in the Linux kernel source tree. DKMS enables kernel device drivers to be automatically rebuilt when a new kernel is installed. An essential feature of DKMS is that it automatically recompiles all DKMS modules if a new kernel version is installed. This allows drivers and devices outside of the mainline kernel to continue working after a Linux kernel upgrade. Another benefit of DKMS is that it allows the installation of a new driver on an existing system, running an arbitrary kernel version, without any need for manual compilation or precompiled packages provided by the vendor.

BSD

Terminal

Basics

See also Typography#Terminal

Xterm

  • How to change the title of an xterm - This document explains how to use escape sequences to dynamically change window and icon titles of an xterm. Examples are given for several shells, and the appendix gives escape sequences for some other terminal types.

Urxvt

if not using a tiling window manager, tabbed is good + remote tmux. urxvt has overhead because of unicode. 

urxvtd -q -f -o
  • Arch Forum: [Solved Change urxvt's font on the fly: Can YOU do it?]
 printf '\33]50;%s\007' "xft:Terminus:pixelsize=16"

! :Bind C-0, C-+ and C-= to activate small, medium, and big font size resp.     
URxvt.keysym.C-0: command:\033]710;-*-dina-medium-r-normal--13-*-*-*-*-*-*-*\007
URxvt.keysym.C-minus: command:\033]710;-*-dina-medium-r-normal--15-*-*-*-*-*-*-*\007
URxvt.keysym.C-equal: command:\033]710;-*-dina-medium-r-normal--16-*-*-*-*-*-*-*\007

Screen

config goes in ~/.screenrc 

escape ^Ww
  change escape key to w

Tmux

Better than screen, if available.

Config goes in ~/.tmux.conf, which can be symlinked to a hidden git repo folder.

Articles

Commands

tmux lsc
  list clients

tmux detach-client -t /dev/pts/26
  remove other clients from session (if screensize is fucked)

split-window

Additions

  • wemux enhances tmux to make multi-user terminal multiplexing both easier and more powerful. It allows users to host a wemux server and have clients join in either:

Misc

Clipboard

The PRIMARY selection is used when you select some text with the mouse. You usually paste it using the middle button. The CLIPBOARD selection is used when you copy text by using, for example, the Edit/Copy menu. You may paste it using the Edit/Paste menu. 

Shift-Insert
  paste clipboard selection (where ctrl-v isn't supported)

Mouse

  • gpm is the mouse support for Linux on the console

Unicode

mirroring char in brackets: (‮‮test (

Shell

/etc/shells

to find;

  • way of making previous command screen output be pushed to a buffer that can be flipped through/forked.
  • paste pwd to readline or whatnot

Bash

man: echo

Basics

Options

Etc.

$_
  last entered word

$?
  returned exit code of last exec

Zsh

Configuration

  • zsh-syntax-highlighting.zsh

lshell

  • lshell is a shell coded in Python, that lets you restrict a user's environment to limited sets of commands, choose to enable/disable any command over SSH (e.g. SCP, SFTP, rsync, etc.), log user's commands, implement timing restriction, and more.

git-shell

  • git-shell - Restricted login shell for Git-only SSH access

Inferno

  • Inferno is a distributed operating system, originally developed at Bell Labs, but now developed and maintained by Vita Nuova® as Free Software. Applications written in Inferno's concurrent programming language, Limbo, are compiled to its portable virtual machine code (Dis), to run anywhere on a network in the portable environment that Inferno provides. Unusually, that environment looks and acts like a complete operating system.

Fish

Other

  • Xiki: A shell console with GUI features. Xiki does what shell consoles do, but lets you edit everything at any time. It's trivial to make your own commands and menus to access other tools. [5]

Files

File systems

Ext

kjournald is responsible for the journal of ext3 [6]

Btrfs

See also Backup#Btrfs

General

Subvolumes appear like directories. inode is different.

"Btrfs support is included in the linux package (as a module). Needs a reboot after installing before btrfs recognised. User space utilities are available in btrfs-progs. For multi-devices support (RAID like feature of btrfs) aka btrfs volume in early boot, you have to enable btrfs mkinitcpio hook (provided by mkinitcpio package) to be able to use, for example, a root btrfs volume. If the btrfs volume is a non-system volume, one only needs to set USEBTRFS="yes" in /etc/rc.conf. However, if you only use bare btrfs partition, such options are not needed."

"The btrfs scrub command reads redundant data and validates all the checksums, correcting any errors it finds along the way, using the checksum to determine which copy is the valid one. But with a single drive, how can it correct anything? The metadata - the file system overhead that is used to manage your data - is always stored in a redundant manner by default, even on a single drive. As a result, any corrupted metadata can be corrected, on the fly."

"EXT4 checksums its journal, which AFAIK will protect against errors caused by sync failures (ie. power failure during disk I/O). But it’s not going to protect against latent sector errors. To do that, you need checksumming on all the file data, along the lines of what ZFS or BTRFS provides."

A cross-subvolume copy patch has made it into 3.6_rc. This patch will allow cp --reflink across subvolumes, as long as the copy does not cross mount points.

Commands
mkfs.btrfs -L [label] /dev/[device]

btrfs filesystem df /media/[drive]
btrfs filesystem show

btrfs-debug-tree -R /dev/sdg
  show drive/subvolume infos, unmounted

btrfs subvolume create [<dest>/]
btrfs subvolume snapshot /mnt/btrfs /mnt/btrfs/snapshot_of_root
btrfs subvolume delete [<dest>/]

mount -o remount,compress=lzo /dev/sdXY /mnt/target

btrfs filesystem defragment /

btrfs device add /dev/sdc /mnt/btrfs

Cloning a file between subvolumes; 

cp --reflink /mnt/MYFILES/myfile1 /mnt/MYFILES/myfile3

copy-on-write, without the ram requirement of zsf snapshots every 30 seconds, ability to mount from previous gen

GUI
  • btrfs-gui is a graphical user interface tool for inspecting and managing btrfs filesystems. It is capable of managing filesystems on the local machine, and filesystems on remote network-accessible machines. It requires root access to the machine to perform most of its tasks (but separates the root-access part from the GUI).
  • Snapper is a tool for managing btrfs snapshots. Apart from the obvious creation and deletion of snapshots it can compare snapshots and revert differences between snapshots. In simple terms, this allows users to view older versions of files and revert changes. Snapper is available as a command line interface tool and a YaST module. Both make use of the C++ library libsnapper which is also available to other programs.
Articles

Ceph

  • Ceph is a free software distributed file system. Ceph's main goals are to be POSIX-compatible, and completely distributed without a single point of failure. The data is seamlessly replicated, making it fault tolerant. Clients mount the file system using a Linux kernel client. On March 19, 2010, Linus Torvalds merged the Ceph client for Linux kernel 2.6.34 which was released on May 16, 2010. An older FUSE-based client is also available. The servers run as regular Unix daemons.

Other

  • Opendedup Develops SDFS, a file-system that does inline deduplication.

RAID

LVM

ZFS

"FreeBSD ZFS tuning guide wiki indicates you'll need about 5GB of ram per 1TB of saved disk space"

File structures

See LSB, etc.

Mount

lsbkl
cat /proc/partitions
blkid

sd[a,b,etc]
  drive
sda[1,2,etc]
  partition of drive

mount /dev/sdxY /some/directory

umount /some/directory

mount -o remount /
  remount partition after /etc/fstab change

Partitions

  • GNU Parted manipulates partition tables. This is useful for creating space for new operating systems, reorganizing disk usage, copying data on hard disks and disk imaging. The package contains a library, libparted, as well as well as a command-line frontend, parted, which can also be used in scripts.
  • http://gparted.sourceforge.net/

dd

  • dd - Copy a file, converting and formatting according to the options.
    • dd is a common Unix program whose primary purpose is the low-level copying and conversion of raw data.
dd if=/dev/sr0 of=myCD.iso bs=2048 conv=noerror,sync
  create an ISO disk image from a CD-ROM.
dd if=/dev/sda2 of=/dev/sdb2 bs=4096 conv=noerror
  Clone one partition to another
dd if=/dev/ad0 of=/dev/ad1 bs=1M conv=noerror
  Clone a hard disk "ad0" to "ad1".

dd if=/dev/zero bs=1024 count=1000000 of=file_1GB
dd if=file_1GB of=/dev/null bs=64k
  drive benchmark test and analyze the sequential read and write performance for 1024 byte blocks

du (disk usage)

du -sh
  size of a folder
du -S
  size of files in a folder

du -aB1m|awk '$1 >= 100'
  everything over 100Mb

cd / | sudo du -khs *
  show root folder size

sudo du -a --max-depth=1 /usr/lib | sort -n -r | head -n 20
  size of program folders /usr/lib

du -sk ./* | sort -nr | awk 'BEGIN{ pref[1]="K"; pref[2]="M"; pref[3]="G";} { total = total + $1;
x = $1; y = 1;  while( x > 1024 ) { x = (x + 1023)/1024; y++; }
printf("%g%s\t%s\n",int(x*10)/10,pref[y],$2); } END { y = 1; while( total > 1024 )
{ total = (total + 1023)/1024; y++; } printf("Total: %g%s\n",int(total*10)/10,pref[y]); }'

ncdu

  • ncdu - ncurses disk usage
ncdu / --exclude /home --exclude /media --exclude /run/media
  check everything apart from home and external drives

ncdu / --exclude /home --exclude /media --exclude /run/media
  check everything apart from external drives

ncdu / --exclude /home --exclude /media --exclude /run/media --exclude /boot
--exclude /tmp --exclude /dev --exclude /proc
 just the root partition

df

  • df - report file system disk space usage

Baobab

Other

todo; source aliases.zsh

  • Filelight creates an interactive map of concentric, segmented rings that help visualise disk usage on your computer.

Disk activity

badly sorted

Files and directories

ls
  list in row
ls -l
  long list

ls *
  files in directory and immediate subdiretories

just names; 

ls -m1
  -m fill width with a comma separated list of entries ??
ls --format single-column
  column of names only
ls -l | grep - | awk '{print $9}'
  using awk to show the 9th word (name). strips colour.
ls -l | cut -f9 -s -d" "
  using cut to cut from the 9th word, using space as a delimiter. strips colour.
ls | cat
  neat

ls -a
  show hidden files
ls  -A
  show hidden files, exclude . and ..

stat .
  display file or file system status
stat -c "%n %a" * | column -t
  directory files + octal

mkdir directory
mkdir directory -p
  no error if existing, make parent directories as needed

ln -s {target-filename}
ln -s {target-filename} {symbolic-filename}
  create soft link

find . -type f -links +1
  list hard links

cd change/directory/path

[8] 

mv position1 ~/position2
  basic move

pax - read and write file archives and copy directory hierarchies 

find . -type f -print0 | xargs -0 stat -c "%y %s %n"

v def conf       =>     vim /some/awkward/path/to/type/default.conf
j abc            =>     cd /hell/of/a/awkward/path/to/get/to/abcdef
m movie          =>     mplayer /whatever/whatever/whatever/awesome_movie.mp4
o eng paper      =>     xdg-open /you/dont/remember/where/english_paper.pdf
vim `f rc lo`    =>     vim /etc/rc.local
vim `f rc conf`  =>     vim /etc/rc.conf

alias defaults; 

alias a='fasd -a'        # any
alias s='fasd -si'       # show / search / select
alias d='fasd -d'        # directory
alias f='fasd -f'        # file
alias sd='fasd -sid'     # interactive directory selection
alias sf='fasd -sif'     # interactive file selection
alias z='fasd_cd -d'     # cd, same functionality as j in autojump
alias zz='fasd_cd -d -i' # cd with interactive selection

File types

xdg-mime default Thunar.desktop inode/directory
  to make Thunar the default file-browser
xdg-mime default xpdf.desktop application/pdf
  to use xpdf as the default PDF viewer

/usr/share/applications/defaults.list      (global)
~/.local/share/applications/defaults.list  (per user, overrides global)

[Default Applications]
mimetype=desktopfile1;desktopfile2;...;desktopfileN

Copying files

cp - copy files and directories

scp -P 2264 foobar.txt your_username@remotehost.edu:/some/remote/directory
scp -rP 2264 folder your_username@remotehost.edu:/some/remote/directory

wget -O myzip.zip https://github.com/zeromq/jzmq/zipball/master
wget -k
  --convert-links

Viewing files

cat

cat filename
  output file to screen
cat -n filename
  output file to screen w/ line numbers
cat filename1 filename2
  output two files (concatinate)
cat filename1 > filename2
  overwrite filename2 with filename1
cat filename1 >> filename2
  append filename1 to filename2
cat filename{1,2} > filename2
  add filename1 and filename2 together into filename3

etc

head filename
  top 10 lines of file
head -23 filename
  top 23 lines of file
tail filename
  bottom 10 lines of file
tail -23 filename
  bottom 23 lines of file

more is a filter for paging through text one screenful at a time. This version is especially primitive. Users should realize that less(1) provides more(1) emulation plus extensive enhancements.

less 

sed -n 20,30p filename
  print lines 20..30 of file [10]

Finding files

find /usr/share -name README
find ~/Journalism -name '*.txt'
find ~/Programming -path '*/src/*.c'

find ~/Images/Screenshots -size +500k -iname '*.jpg'
find ~/Journalism -name '*.txt' -exec cat {} ;
find ~/Journalism -name '*.txt' -print0 | xargs -0 cat   (faster than above)

find / -group [group]
find / -user [user]

find . -mtime -[n]
  File's data was last modified n*24 hours ago

http://arstechnica.com/information-technology/2011/07/ask-ars-how-to-use-the-find-command-in-a-pipeline/

Finding file content

grep

grep "stuff" sqldump.sql | fold -w 200 | grep -C 1 "stuff"

The first grep gets the (mile-wide) line that has the match, then fold will split the mile-wide line into 200 char long lines, and "grep -C 1" will show only the one 200 char wide line where the match is + 1 line of context before and after. [11]

  • sgrep - search a file for a structured pattern

Compression

tar

tar <operation> [options]
Operations:
  [-]A --catenate --concatenate
  [-]c --create
  [-]d --diff --compare
  [-]r --append
  [-]t --list
  [-]u --update
  [-]x --extract --get
  --delete

Common Options:
  -C, --directory DIR
  -f, --file F
  -j, --bzip2
  -p, --preserve-permissions
  -v, --verbose
  -z, --gzip

zip

gzip
bzip
7z
  • 7-Zip is a file archiver with the highest compression ratio. The program supports 7z (that implements LZMA compression algorithm), ZIP, CAB, ARJ, GZIP, BZIP2, TAR, CPIO, RPM and DEB formats. Compression ratio in the new 7z format is 30-50% better than ratio in ZIP format.
    • p7zip is a port of 7za.exe for POSIX systems like Unix (Linux, Solaris, OpenBSD, FreeBSD, Cygwin, AIX, ...), MacOS X and also for BeOS and Amiga. 7za.exe is the command line version of 7-zip, see http://www.7-zip.org/. 7-Zip is a file archiver with highest compression ratio.
    • man z7 (p7zip)
    • p7zip-light in AUR
7z x filename
  extract archive with directories
xz

pax

  • pax will read, write, and list the members of an archive file, and will copy directory hierarchies. pax operation is independent of the specific archive format, and supports a wide variety of different archive formats. A list of supported archive formats can be found under the description of the -x option. [12]

Other

Generic function

# Extract Files
extract() {
 if [ -f $1 ] ; then
     case $1 in
         *.tar.bz2)   tar xvjf $1    ;;
         *.tar.gz)    tar xvzf $1    ;;
         *.tar.xz)    tar xvJf $1    ;;
         *.bz2)       bunzip2 $1     ;;
         *.rar)       unrar x $1     ;;
         *.gz)        gunzip $1      ;;
         *.tar)       tar xvf $1     ;;
         *.tbz2)      tar xvjf $1    ;;
         *.tgz)       tar xvzf $1    ;;
         *.zip)       unzip $1       ;;
         *.Z)         uncompress $1  ;;
         *.7z)        7z x $1        ;;
         *.xz)        unxz $1        ;;
         *.exe)       cabextract $1  ;;
         *)           echo "\`$1': unrecognized file compression" ;;
     esac
 else
     echo "\`$1' is not a valid file"
 fi
}

Commands

Resources

Use output as input. 

cat `locate file.txt`

or 

cat $(locate file.txt)

todo; piping and redirection

  • inotail is a replacement for the 'tail' program found in the base installation of every Linux/UNIX system. It makes use of the inotify infrastructure in recent versions of the Linux kernel to speed up tailing files in the follow mode (the '-f' option). Standard tail polls the file every second by default while inotail listens to special events sent by the kernel through the inotify API to determine whether a file needs to be reread.

Currently inotail is not fully compatible to neither POSIX or GNU tail but might be in the future.

Configuration

/etc
~/.config

etc.

Daemons

A daemon is a program that runs in the background, waiting for events to occur and offering services. A good example is a web server that waits for a request to deliver a page or a ssh server waiting for someone trying to log in. While these are full featured applications, there are daemons whose work is not that visible. Daemons are for tasks like writing messages into a log file (e.g. syslog, metalog) or keeping your system time accurate (e.g. ntpd). For more information see man 7 daemon.

Programs

  • auditd - userspace component to the Linux Auditing System [15]
  • nice - run a program with modified scheduling priority
  • cut - remove sections from each line of files
  • shelr - console screencasting tool

~/.local/share/applications/mimeinfo.cache

Finding programs

whereis

apropos apropos vim
  search the whatis database for strings

Monitoring

See also Server#Monitoring

lsof

  • lsof - "lists open files" (lots, given "everything" is a file)
lsof -i :[port]
  what application using a specific port

top

Part of the procps package.

Current stats only, no disk or network. Sorts by cpu by default. A multicore box can show more than 100%. 

z
  colour!
d
  change update interval
return, space
  refresh

H
  show separate threads of each process
i
  show run queue (R) or waiting on disk io (D, red)
1
  show serperate cpus

q
  quit

htop

atop

Shows short lived processes that run and finish between update interval times.

lines;

  • PRC: Total CPU time in system and user mode, total number of processes and of zombie processes, and the number of processes that exited during the polling interval.
  • CPU and CPL: CPU utilization and load (averaged over 1, 5 and 15 minutes).
  • cpu: individual CPU usage.
  • MEM and SWP: Amount of memory and swap space that is available and where it’s allocated. vmcom and vmlim show how much virtual memory space is committed and what the limit is.
  • DSK: disk utilization. avio shows the average number of milliseconds per request.
  • NET: Network utilization for the TCP layer (“transport”), the IP layer (“network”) and each interface.
Ctrl-f
  forward a page
Ctrl-b
  back a page

C
  sort by cpu activity
M
  sort by memory consumption
D
  sort by disk activity
N
  sort by network activity
A
  sort by most active system resource (auto mode)

P
  focus on process (regex)
y
  show all threads (inc. idle)

s
 Scheduling information
m
  Memory consumption
d
  Disk utilization
v
 Variable information
c
  Command line
p
 Accumulated per program
u
  Accumulated per user
n
 Network utilization (patched kernel)

i
  interval timer
t
  trigger update

other

  • free - Display amount of free and used memory in the system
  • vpsinfo is a Linux server monitoring script, written in PHP, that provides web access to system status information. It gathers the output from several common Linux commands into one web page, providing a quick overview of the system's current state.

Cron

A cronjob is a task that a Cron system is instructed to run periodically. The crontab file is a configuration file for a user that defines tasks to run under the user’s account. The systab file is a file that specifies cronjobs for the system.

Config

crontab -l
  view crontabs
crontab -e
  edit their crontabs
crontab -r
 remove their crontabs
crontab saved_crontab_filename
  overwrite their old crontab with saved crontab

There are several special predefined values which can be used to substitute the CRON expression. 

Entry                  Description                                 Equivalent To
 @yearly (or @annually) Run once a year, midnight, Jan. 1st         0 0 1 1 *
 @monthly               Run once a month, midnight, first of month  0 0 1 * *
 @weekly                Run once a week, midnight on Sunday         0 0 * * 0
 @daily                 Run once a day, midnight                    0 0 * * *
 @hourly                Run once an hour, beginning of hour         0 * * * *
 @reboot                Run at startup                              @reboot 

*    *    *    *    *  command to be executed
┬    ┬    ┬    ┬    ┬
│    │    │    │    │
│    │    │    │    │
│    │    │    │    └───── day of week (0 - 6) (0 is Sunday, or use names)
│    │    │    └────────── month (1 - 12)
│    │    └─────────────── day of month (1 - 31)
│    └──────────────────── hour (0 - 23)
└───────────────────────── min (0 - 59)
  • /etc/cron.allow - If this file exists, then you must be listed therein (your username must be listed) in order to be allowed to use cron jobs.
  • /etc/cron.deny - If the cron.allow file does not exist but the /etc/cron.deny file does exist, then you must not be listed in the /etc/cron.deny file in order to use cron jobs.

dcron

Vanilla dcron supports asynchronous job processing. Just put it with @hourly, @daily, @weekly or @monthly with a jobname.

dcron has @daily, @hourly and so on. In fcron, you can use standard crontab entries and add &bootrun to the beginning of the line to repeat "missed" cronjobs.

cronie

Arch default cron as dcron wasn't maintained at the time.

dcron and fcron are not under active development, cronie is. cronie is small - 0.20MB installed. developed by Red Hat - it is not going anywhere and we have a guaranteed upgrade path. As far as I can tell cronie has no deps beyond glibc and pam. cronie has /etc/cron.d support and anacron support via an anacrontab config file. cronie extends the original vixie cron package so the syntax, core feature set, etc are stable. implements advanced security hooks as well and can integrate with SELINUX.

anacron

  • anacron is a computer program that performs periodic command scheduling which is traditionally done by cron, but without assuming that the system is running continuously. Thus, it can be used to control the execution of daily, weekly, and monthly jobs (or anything with a period of n days) on systems that don't run 24 hours a day.

fcron

Fcron’s own crontab system uses the fcrontab file for configuration information. The fcrontab syntax is similar but differs slightly from the classic Vixie/ISC Cron crontab notation. fcron lacks /etc/cron.d/ functionality.

Wildcards

MIME

chroot

Startup

  • e4rat ("Ext4 - Reducing Access Times") is a toolset to accelerate the boot process as well as application startups. Through physical file realloction e4rat eliminates both seek times and rotational delays. This leads to a high disk transfer rate. Placing files on disk in a sequentially ordered way allows to efficiently read-ahead files in parallel to the program startup.

ncurses, etc.

Users

  • /etc/passwd - local user information
    • account:password:UID:GID:GECOS:directory:shell
  • /etc/shadow - restricted access encrypted password file
useradd username
  create user with defaults (no password)
useradd -D
  show defaults that will be used

useradd -m -g [initial_group] -G [additional_groups] -s [login_shell] [username]
  -m - add home dir
  -d - non-default home dir path
  -c comment
  -e 2006-06-30 - expires
useradd -m -g users -G audio,lp,optical,storage,video,wheel,games,power,scanner -s /bin/bash username

adduser
  interactive tool

passwd
  set password of current account
passwd username
  set password of username
passwd -S username
  see password status of username

userdel username
  remove user
userdel -r username
  remove user plus their home folder, mail spool

cat /etc/passwd | cut -d: -f 1,3,6 | grep "[5-9][0-9][0-9]"
  List users on a system [17]
  • last, lastb - show listing of last logged in users

Groups

groupadd [group]
  add group

gpasswd -a [user] [group]
  add user to group
gpasswd -d [user] [group]
  delete user from group

User must relogin for new group to take effect.

Permissions

stat -c '%A %a %n' *
  list permissions in octal

chmod
  change file mode bits

chown -R user:group .
  change all and subitems [18]

su / sudo

Run things as root or another user. There are a number of methods to do higher privilege things, with differing resultant permissions. To refind good forum threads..

snippet to source... 

sudo sh -c "echo 'something' >> /etc/privilegedfile"

sudoers

visudo
  command to safely add sudoers rights using nano

user ALL=(ALL) ALL

ACLs

Partition must have acl set in /etc/fstab (and be remounted after). 

setfacl -m "u:username:permissions"
setfacl -m "u:uid:permissions"
  add permissions for user

setfacl -m "g:groupname:permissions"
setfacl -m "g:gid:permissions"
  add permissions for group

setfacl -m "u:user:rwx" file
  add read, write, execure perms for user for file
setfacl -Rm "u:user:rw" /dir
  add recursive read, write perms for user for dir
setfacl -Rdm "u:user:rw" /dir
  add recursive read, write perms for user for dir and make them default for future changes

Session permission management

For non-root mount permissions, etc.

Networking

Service commands

sudo /etc/init.d/<service> restart
  ubuntu, restart a service

sudo /etc/rc.d/<service> stop | start | restart
  arch, service things

route
  show network routes avaliable
route -n
  show network routes avaliable, just ip

Configuration

mtr
  • route -n - display the host's networks and gateway

to sort

  • View The Daily Show, etc. in the UK, etc. Mofity HTTP headers; X-Forwarded-For "12.13.14.15" [20]

curl

  • curl is a tool to transfer data from or to a server, using one of the supported protocols (DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and TFTP). The command is designed to work without user interaction.
curl http://www.google.com/search.js -o /path/to/local/file.js

curl http://site.{one,two,three}.com

curl ftp://ftp.numericals.com/file[1-100].txt
     ftp://ftp.numericals.com/file[001-100].txt (with leading zeros)
     ftp://ftp.letters.com/file[a-z].txt
 sequences of alphanumeric series by using []

curl http://any.org/archive[1996-1999]/vol[1-4]/part{a,b,c}.html
  Nested sequences are not supported, but you can use several ones next to each other:

curl http://www.numericals.com/file[1-100:10].txt  http://www.letters.com/file[a-z:2].txt
  multiple urls + specify a step counter for the ranges to get every Nth number or letter:

ftp

nmap

nmap -sT -sU -O -p 1-65535 localhost
  full port scan

ntop

vnStat

  • vnStat is a console-based network traffic monitor for Linux and BSD that keeps a log of network traffic for the selected interface(s). It uses the network interface statistics provided by the kernel as information source. This means that vnStat won't actually be sniffing any traffic and also ensures light use of system resources. However, in Linux at least a 2.2 series kernel is required.

VOIP

http://www.itworld.com/print/166637

  • Karaka is a Skype/XMPP gateway that connects the Skype and XMPP clouds.

NFS

File sharing

Media

Package management

Apt

dpkg --get-selections > installed-software
  create list of installed software

dpkg --set-selections < installed-software
dselect
  reinstall from list

dpkg --list
  to check

Pacman

pacman -Sy [package]
  install [package]
pacman -Syu
  upgrade system

pacman -Qo [file]
  check what package owns a file
pacman -Qi
  list installed programms

pacman -Qqtd
  check whether there are any orphaned packages
pacman -Rsn packagename
  remove orphaned packages
pacman -Rc packagename
  remove package and deps ('cascade')
  • cacheclean - Cleans up pacman packages. Users selects how many old versions to keep.
cacheclean {-p} {-v} <# of copies to keep>
# of copies to keep - (required) how many generations of each package.
-p - (optional) preview what would be deleted.
-v - (optional) show deleted packages.

for i in `pacman -Qdt | awk '{print $1}'`; do pacman -R $i ; done

Helpers

Packages

 makepkg -g >> PKGBUILD && makepkg

"ERROR: PKGBUILD contains CRLF characters and cannot be sourced." Solution: 

sed -i 's/^M//' PKGBUILD

[ctrl+v][ctrl+m] for the ^M symbol. 

pacman -Qdt
  list all orphans
pacman -Rsn $(pacman -Qdtq)
  remove them all

Repos

Other

  • gist: unused.sh: finds unused package on your Arch Linux box

Other

Documentation

man

Time

echo "Today's date is `date -I`"

date
  Local datetime
date -d @1337000000
  Convert unixtime to datetime
date -ud
  Convert unixtime to UTC datetime

ntpdate pool.ntp.org && hwclock --systohc && hwclock --adjust
  Synchronize both your system clock and hardware clock and calculate/adjust time drift.
  Do not run this command if you already have ntpd running! [22]

Swap

swapon -s
free -m

Suspension, hibernation

Other software

CUPS

Printing system.

Cisco

Storage

Plan 9

Unix done 'better'. Ideas from Plan 9 made it into Linux, like more things being files.

Inferno

Retrieved from "https://wiki.thingsandstuff.org/index.php?title=*nix&oldid=6186"