Server
Service commands
sudo /etc/init.d/<service> restart ubuntu, restart a service sudo /etc/rc.d/<service> stop | start | restart arch, service things - old, use systemd now
General
- https://wiki.archlinux.org/index.php/Comprehensive_Server_Guide
- http://arstechnica.com/gadgets/2012/11/how-to-set-up-a-safe-and-secure-web-server/
- http://en.wikipedia.org/wiki/CAP_theorem
- http://en.wikipedia.org/wiki/Fallacies_of_Distributed_Computing
- http://en.wikipedia.org/wiki/Software_as_a_service
- http://en.wikipedia.org/wiki/Platform_as_a_service
- http://en.wikipedia.org/wiki/Cloud_computing
Sysadmin
Logging
Basics
- Linux log files location and how do I view logs files? - Jul 17, 2006
- 20 Linux Log Files that are Located under /var/log Directory - Aug 1, 2011
- HowTo: The Ultimate Logrotate Command Tutorial with 10 Examples - Jul 14, 2010
- logrotate is designed to ease administration of systems that generate large numbers of log files. It allows automatic rotation, compression, removal, and mailing of log files. Each log file may be handled daily, weekly, monthly, or when it grows too large.
- Using logrotate to rotate and archive log - Aug 05, 2010
Systems
syslog
- syslog
- syslogd
- /etc/syslog.conf
- How To Set Up A Debian Linux Syslog Server
- logger - a shell command interface to the syslog(3) system log module
syslog-ng
- syslog-ng is a high-performance syslog server with advanced log processing services and direct database access. syslog-ng clients collect the log messages from the various applications, files, and other sources then send important log messages to the remote syslog-ng server which sorts and stores them.
rsyslog
- Rsyslog is an enhanced syslogd supporting, among others, MySQL, PostgreSQL, failover log destinations, syslog/tcp, fine grain output format control, high precision timestamps, queued operations, the ability to filter on any message part and encryption protected syslog relay chains. It is quite compatible to stock sysklogd and can be used as a drop-in replacement.
journald
systemd's journal service
- http://www.freedesktop.org/software/systemd/man/systemd-journald.service.html
- http://www.freedesktop.org/software/systemd/man/journald.conf.html
- https://wiki.archlinux.org/index.php/Systemd#Journal
- https://docs.google.com/document/pub?id=1IC9yOXj7j6cdLLxWEBAGRL6wl97tFxgjLUEHIX3MSTs
other
- glTail.rb - Real-time visualization of server traffic, events and statistics with Ruby, SSH and OpenGL.
- log.io Real-time log monitoring in your browser. Powered by node.js + socket.io
- logstash is a tool for managing events and logs. You can use it to collect logs, parse them, and store them for later use (like, for searching). Speaking of searching, logstash comes with a web interface for searching and drilling into all of your logs.
- Dendrite scrapes your existing logs and re-emits a unified log stream in modern, sensible, structured formats like JSON and StatsD over common protocols such as TCP, UDP, file streams
- Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. It has a simple and flexible architecture based on streaming data flows. It is robust and fault tolerant with tunable reliability mechanisms and many failover and recovery mechanisms. It uses a simple extensible data model that allows for online analytic application.
- Fluentd is an open-source tool to collect events and logs. 150+ plugins instantly enables you to store the massive data for Log Search, Big Data Analytics, and Archiving (MongoDB, S3, Hadoop).
- Kibana is an open source (Apache Licensed), browser based analytics and search interface to Logstash and other timestamped data sets stored in ElasticSearch. With those in place Kibana is a snap to setup and start using (seriously). Kibana strives to be easy to get started with, while also being flexible and powerful
Services
Monitoring
See also Network#Monitoring, *nix#Monitoring
If it's up... | A.K.A... | It's down... per year |
---|---|---|
90% | n/a | 876 hours |
95% | n/a | 438 hours |
99% | two 9s | 87 hours, 36 minutes |
99.9% | three 9s | 8 hours, 45 minutes, 36 seconds |
99.99% | four 9s | 52 minutes, 33.6 seconds |
99.999% | five 9s | 5 minutes, 15.36 seconds |
99.9999% | six 9s | 31.68 seconds |
Collectd
collectd is a daemon which collects system performance statistics periodically and provides mechanisms to store the values in a variety of ways, for example in RRD files. collectd gathers statistics about the system it is running on and stores this information. Those statistics can then be used to find current performance bottlenecks (i.e. performance analysis) and predict future system load (i.e. capacity planning). Or if you just want pretty graphs of your private server and are fed up with some homegrown solution you're at the right place, too ;).
Nagios
- Nagios is a powerful monitoring system that enables organizations to identify and resolve IT infrastructure problems before they affect critical business processes. Designed with scalability and flexibility in mind, Nagios gives you the peace of mind that comes from knowing your organization's business processes won't be affected by unknown outages.
Monit
- Monit is a free open source utility for managing and monitoring, processes, programs, files, directories and filesystems on a UNIX system. Monit conducts automatic maintenance and repair and can execute meaningful causal actions in error situations.
- M/Monit expand upon Monit's capabilities to provide monitoring and management of all Monit enabled hosts from one easy to use web-interface.
Splunk
- Splunk is powerful and versatile IT search software that takes the pain out of tracking and utilizing the information in your data center. If you have Splunk, you won't need complicated databases, connectors, custom parsers or controls--all that's required is a web browser and your imagination. Splunk handles the rest.
- YouTube: Splunk 4.3 Overview
Amon
- Amon is an application and system monitoring toolkit for web developers. Its goal is to provide you with all the information you need about the health of your servers and the errors that occur in your web application.
Munin
Shinken
Sensu
other
- http://aphyr.github.com/riemann/ - distributed
- https://github.com/RaymiiOrg/raymon/ - PHP, Bash, JSON
Zenoss
Services
New Relic
Rackspace
Management
- Webmin is a web-based interface for system administration for Unix. Using any modern web browser, you can setup user accounts, Apache, DNS, file sharing and much more. Webmin removes the need to manually edit Unix configuration files like /etc/passwd, and lets you manage a system from the console or remotely.
- Virtualmin is a powerful and flexible web hosting control panel for Linux and UNIX systems based on the well-known Open Source web-based systems management GUI, Webmin. Manage your virtual domains, mailboxes, databases, applications, and the entire server, from one comprehensive and friendly interface.
- Ehcp is a Hosting Control Panel to facilate the process of managing domains, e-mail, users, etc.
- AlternC is a set of software management on Linux shared hosting, easy to use, easy to install, based solely on free software, GPL v2 +. Systems for automatic installation and configuration, control panel accessible via the web for the management of users and web services (domains, emails, ftp accounts, statistics ...). Technically, AlternC is currently based on Debian GNU / Linux 'Stable' (Squeeze to date), and use other software such as Apache, Postfix, Mailman ... It also contains API documentation, allowing easy customization of the office, 'web control panel' AlternC.
- easyengine (ee) is a Linux shell-script collection, which makes it easy to manage your WordPress-Nginx sites on an Ubuntu server
Performance
See also HTTP#Performance
- irqbalance is a Linux* daemon that distributes interrupts over the processors and cores you have in your computer system. The design goal of irqbalance is to do find a balance between power savings and optimal performance. To a large degree, the work irqbalance does is invisible to you; if irqbalance performs its job right, nobody will ever notice it's there or want to turn it off.
- HAProxy - The Reliable, High Performance TCP/HTTP Load Balancer.
- Pound is a reverse proxy - that means you put it on the server in front of your web services, not in front of your clients who need to connect to the general Internet. It takes web requests from end-users and distributes them among several web servers or services you may be running. Pound is also load balancing, so you can run multiple servers that look to the outside world as if they are just one, allowing you to spread the workload around. Pound proxy is NOT a caching proxy. By itself it won't help to speed up your server or network, but there is a lot of flexibility in Pound that will help you overall.
memcached
- Memcached is a free & open source, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load. Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering. Memcached is simple yet powerful. Its simple design promotes quick deployment, ease of development, and solves many problems facing large data caches. Its API is available for most popular languages.
- magent is a simple but useful proxy program for memcached servers.
Redis
See also DB#Redis
- http://www.quora.com/What-are-the-differences-between-memcached-and-redis
- http://systoilet.wordpress.com/2010/08/09/redis-vs-memcached/
- http://helpdesk.getpantheon.com/customer/portal/articles/401317-redis
CDN Services
Other
Filesystem
Knowledge management
Video streaming
Media
See also Platforms#Digital_media
Push/pull
Superfeedr
Storage
WebDAV
- WebDAV stands for Web Distributed Authoring and Versionin, see RFC 2518
"Linux users can mount WebDAV shares using the davfs2 and the fusedav file system modules which mount them as Coda or FUSE filesystems. KDE has native WebDAV support as part of kio_http. This enables Dolphin, Konqueror, and every other KDE application to interact directly with WebDAV servers. Nautilus also has WebDAV support built in. Many Linux distributions also include the cadaver command-line client interface, which provides an FTP-like command set. The Apache HTTP Server provides WebDAV modules based on both davfs and Apache Subversion (svn)."
- WebDAV Resources - This site is being produced for the WebDAV community as a central resource for documentation, specifications, software, mailing lists, and other useful items.
- Use Linux and WebDAV to Facilitate Online Collaboration - Apache method
- http://linuxsagas.digitaleagle.net/2008/09/09/webdav-and-fstab/
- davfs2 provides the ability to access such resources like a typical filesystem, allowing for use by standard applications with no built-in support for WebDAV.
- fusedav is a Linux userspace file system driver for mounting WebDAV shares. It makes use of FUSE as userspace file system API and neon as WebDAV API.
- http://owncloud.org/ - Version 5.0 Expected August 2012
- https://github.com/mikedeboer/jsDAV - node.js
- http://dav-pocket.appspot.com/ - webdav w/ google
Other
- http://www.somethingsimilar.com/2013/01/14/notes-on-distributed-systems-for-young-bloods/
- http://www.extremetech.com/computing/123929-just-how-big-are-porn-sites