Server

From Things and Stuff Wiki
Revision as of 20:07, 28 September 2015 by Milk (talk | contribs) (→‎Basics)
Jump to navigation Jump to search


General

  • Tweaked.io - This site is dedicated to information relating to tweaking, tuning, and optimizing your server-software. Steve Kemp.

Sysadmin

history
history info | awk '{CMD[$2]++;count++;}END { for (a in CMD)print CMD[a] " " CMD[a]/count*100 "% " a;}' | grep -v "./" | column -c3 -s 
w
last -a
uname -a
grep -H  /etc/*release /proc/version
df -h
free -m
who
id
netstat -tlpnu
iptables -nvxL
ss -s
top
ps auxf
dmesg |less
ls -lR /etc/cron*
tail /var/log/*
cat /proc/cpuinfo
lsblk -io KNAME,TYPE,SIZE,MODEL 
findmnt
env

Services

See also *nix#init 

sudo /etc/init.d/<service> restart
  SysV

sudo systemctl restart <service>
  Systemd

Logging

Basics

logrotate

  • logrotate is designed to ease administration of systems that generate large numbers of log files. It allows automatic rotation, compression, removal, and mailing of log files. Each log file may be handled daily, weekly, monthly, or when it grows too large.

Systems

syslog

  • logger - a shell command interface to the syslog(3) system log module

syslog-ng

rsyslog

  • Rsyslog is an enhanced syslogd supporting, among others, MySQL, PostgreSQL, failover log destinations, syslog/tcp, fine grain output format control, high precision timestamps, queued operations, the ability to filter on any message part and encryption protected syslog relay chains. It is quite compatible to stock sysklogd and can be used as a drop-in replacement.

journald

systemd's journal service

Aggregation

  • Logcheck is a simple utility which is designed to allow a system administrator to view the logfiles which are produced upon hosts under their control. It does this by mailing summaries of the logfiles to them, after first filtering out "normal" entries.
  • logstash is a tool for managing events and logs. You can use it to collect logs, parse them, and store them for later use (like, for searching). Speaking of searching, logstash comes with a web interface for searching and drilling into all of your logs.
  • Graylog2 enables you to unleash the power that lays inside your logs. Use it to run analytics, alerting, monitoring and powerful searches over your whole log base. Need to debug a failing request? Just run a quick filter search to find it and see what errors it produced. Want to see all messages a certain API consumer is consuming in real time? Create streams for every consumer and have them always only one click away.
  • GoAccess is an open source real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems. It provides fast and valuable HTTP statistics for system administrators that require a visual server report on the fly.
  • lnav - An advanced log file viewer for the small-scale. Watch and analyze your log files from a terminal. No server. No setup. Still featureful.
  • Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. It has a simple and flexible architecture based on streaming data flows. It is robust and fault tolerant with tunable reliability mechanisms and many failover and recovery mechanisms. It uses a simple extensible data model that allows for online analytic application.
  • Fluentd is an open-source tool to collect events and logs. 150+ plugins instantly enables you to store the massive data for Log Search, Big Data Analytics, and Archiving (MongoDB, S3, Hadoop).
  • log.io - Real-time log monitoring in your browser. Powered by node.js + socket.io

http://www.vanheusden.com/multitail/

  • glTail.rb - Real-time visualization of server log traffic, events and statistics with Ruby, SSH and OpenGL.

Services

Monitoring

See also Network#Monitoring, *nix#Monitoring

If it's up... A.K.A... It's down... per year
90% n/a 876 hours
95% n/a 438 hours
99% two 9s 87 hours, 36 minutes
99.9% three 9s 8 hours, 45 minutes, 36 seconds
99.99% four 9s 52 minutes, 33.6 seconds
99.999% five 9s 5 minutes, 15.36 seconds
99.9999% six 9s 31.68 seconds

lbcd

Collectd

collectd is a daemon which collects system performance statistics periodically and provides mechanisms to store the values in a variety of ways, for example in RRD files. collectd gathers statistics about the system it is running on and stores this information. Those statistics can then be used to find current performance bottlenecks (i.e. performance analysis) and predict future system load (i.e. capacity planning). Or if you just want pretty graphs of your private server and are fed up with some homegrown solution you're at the right place, too ;).

Nagios

  • Nagios is a powerful monitoring system that enables organizations to identify and resolve IT infrastructure problems before they affect critical business processes. Designed with scalability and flexibility in mind, Nagios gives you the peace of mind that comes from knowing your organization's business processes won't be affected by unknown outages.
  • NagVis is a visualization addon for the well known network managment system Nagios.
  • NPC is to be a complete web based UI replacement to Nagios while fully integrating into Cacti using the Cacti Plugin Architecture. The primary benefit is a single point of access for trending and alert monitoring.

Monit

  • Monit is a free open source utility for managing and monitoring, processes, programs, files, directories and filesystems on a UNIX system. Monit conducts automatic maintenance and repair and can execute meaningful causal actions in error situations.
  • M/Monit expand upon Monit's capabilities to provide monitoring and management of all Monit enabled hosts from one easy to use web-interface.

Splunk

  • Splunk is powerful and versatile IT search software that takes the pain out of tracking and utilizing the information in your data center. If you have Splunk, you won't need complicated databases, connectors, custom parsers or controls--all that's required is a web browser and your imagination. Splunk handles the rest.

Amon

  • Amon is an application and system monitoring toolkit for web developers. Its goal is to provide you with all the information you need about the health of your servers and the errors that occur in your web application.

Munin

Shinken

Sensu

Other

  • Cacti is a complete network graphing solution designed to harness the power of RRDTool's data storage and graphing functionality. Cacti provides a fast poller, advanced graph templating, multiple data acquisition methods, and user management features out of the box. All of this is wrapped in an intuitive, easy to use interface that makes sense for LAN-sized installations up to complex networks with hundreds of devices.
  • ApacheTop is a curses-based top-like display for Apache information, including requests per second, bytes per second, most popular URLs, etc.
  • Riemann aggregates events from your servers and applications with a powerful stream processing language. Send an email for every exception raised by your code. Track the latency distribution of your web app. See the top processes on any host, by memory and CPU. Combine statistics from every Riak node in your cluster and forward to Graphite. Send alerts when a key process fails to check in. Know how many users signed up right this second.
  • Custodian allows a wide range of services to be remotely tested across a network. The system uses a central work-queue, which allows the testing to be carried out by a agents running upon an arbitrary number of hosts.
  • Blueflood is a multi-tenant distributed metric processing system created by engineers at Rackspace. It is used in production by the Cloud Monitoring team to process metrics generated by their monitoring systems. Blueflood is capable of ingesting, rolling up and serving metrics at a massive scale.
  • Ray-Mon is a linux server monitoring script written in PHP and Bash, utilizing JSON as data storage. It requires only bash and a webserver on the client side, and only php on the server side.
  • Pinba is a MySQL storage engine that acts as a realtime monitoring/statistics server for PHP using MySQL as a read-only interface. It accumulates and processes data sent over UDP by multiple PHP processes and displays statistics in a nice human-readable form of simple "reports", also providing read-only interface to the raw data in order to make possible generation of more sophisticated reports and stats.
  • Opserver is a monitoring system by the team at Stack Exchange, home of Stack Overflow.
  • Centreon is an free and Open Source software allowing you to monitor all the infrastructures and applications of your information system
  • Icinga is an enterprise grade open source monitoring system which keeps watch over networks and any conceivable network resource, notifies the user of errors and recoveries and generates performance data for reporting. Scalable and extensible, Icinga can monitor complex, large environments across dispersed locations. Icinga is a fork of Nagios and is backward compatible.
  • Zabbix is the ultimate open source availability and performance monitoring solution. Zabbix offers advanced monitoring, alerting, and visualization features today which are missing in other monitoring systems, even some of the best commercial ones.
  • Xymon is a system for monitoring servers and networks. It takes its inspiration from the Big Brother monitoring system, but unlike Big Brother it is designed to work well whether you need to monitor a small network with just a handful of hosts, or large networks with thousands of servers and network services.
  • Ganglia is a scalable distributed monitoring system for high-performance computing systems such as clusters and Grids. It is based on a hierarchical design targeted at federations of clusters. It leverages widely used technologies such as XML for data representation, XDR for compact, portable data transport, and RRDtool for data storage and visualization.
  • Hyperic's web infrastructure monitoring and management software automates and streamlines data center operations. HQ helps you reduce operations workload, increase your company's IT management maturity level, and drive improvements in availability and infrastructure health.
  • Packetbeat - Open Source application monitoring & packet tracing system
  • Inspeqtor monitors your application infrastructure. It gathers and verifies key metrics from all the moving parts in your application and alerts you when something looks wrong. It understands the application deployment workflow so it won't bother you during a deploy.
  • termui - Go terminal dashboard. Inspired by blessed-contrib, but purely in Go. Cross-platform, easy to compile, and fully-customizable.

Zenoss

linux-dash

GoAccess

  • GoAccess is an open source real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems. It provides fast and valuable HTTP statistics for system administrators that require a visual server report on the fly.

Services

New Relic

Rackspace

Web management

  • Webmin is a web-based interface for system administration for Unix. Using any modern web browser, you can setup user accounts, Apache, DNS, file sharing and much more. Webmin removes the need to manually edit Unix configuration files like /etc/passwd, and lets you manage a system from the console or remotely.
  • Virtualmin is a powerful and flexible web hosting control panel for Linux and UNIX systems based on the well-known Open Source web-based systems management GUI, Webmin. Manage your virtual domains, mailboxes, databases, applications, and the entire server, from one comprehensive and friendly interface.
  • Ehcp is a Hosting Control Panel to facilate the process of managing domains, e-mail, users, etc.
  • AlternC is a set of software management on Linux shared hosting, easy to use, easy to install, based solely on free software, GPL v2 +. Systems for automatic installation and configuration, control panel accessible via the web for the management of users and web services (domains, emails, ftp accounts, statistics ...). Technically, AlternC is currently based on Debian GNU / Linux 'Stable' (Squeeze to date), and use other software such as Apache, Postfix, Mailman ... It also contains API documentation, allowing easy customization of the office, 'web control panel' AlternC.

Performance

See also HTTP#Performance

  • irqbalance is a Linux* daemon that distributes interrupts over the processors and cores you have in your computer system. The design goal of irqbalance is to do find a balance between power savings and optimal performance. To a large degree, the work irqbalance does is invisible to you; if irqbalance performs its job right, nobody will ever notice it's there or want to turn it off.
  • HAProxy - The Reliable, High Performance TCP/HTTP Load Balancer.
  • Pound is a reverse proxy - that means you put it on the server in front of your web services, not in front of your clients who need to connect to the general Internet. It takes web requests from end-users and distributes them among several web servers or services you may be running. Pound is also load balancing, so you can run multiple servers that look to the outside world as if they are just one, allowing you to spread the workload around. Pound proxy is NOT a caching proxy. By itself it won't help to speed up your server or network, but there is a lot of flexibility in Pound that will help you overall.

memcached

  • magent is a simple but useful proxy program for memcached servers.

Redis

See also DB#Redis

CDN services

Other

  • OpenNOP is an open source Linux based network accelerator. It's designed to optimise network traffic over point-to-point, partially-meshed and full-meshed IP networks. The goal of OpenNOP is to increase network performance over low speed network connections such as ISDN, DSL, DS1, frame-relay, VPN and other network circuits. To accomplish this we can use a variety of techniques such as compression, deduplication, caching and protocol spoofing.

Filesystem

Configuration management

Video streaming

Media

See also Platforms#Digital_media

Push/pull

Superfeedr

Storage

WebDAV

"Linux users can mount WebDAV shares using the davfs2 and the fusedav file system modules which mount them as Coda or FUSE filesystems. KDE has native WebDAV support as part of kio_http. This enables Dolphin, Konqueror, and every other KDE application to interact directly with WebDAV servers. Nautilus also has WebDAV support built in. Many Linux distributions also include the cadaver command-line client interface, which provides an FTP-like command set. The Apache HTTP Server provides WebDAV modules based on both davfs and Apache Subversion (svn)."

File system

Davfs
fusedav
  • fusedav is a Linux userspace file system driver for mounting WebDAV shares. It makes use of FUSE as userspace file system API and neon as WebDAV API.

Software

  • neon is an HTTP and WebDAV client library, with a C interface
  • jsDAV allows you to easily add WebDAV support to a NodeJS application.


  • DAV-pocket Lab is a small project team developping WebDAV Server on Google App Engine.


  • mod_dav is an Apache module to provide DAV capabilities (RFC 2518) for your Apache web server.
  • http://www.webdav.org/goliath/
  • cadaver is a command-line WebDAV client for Unix. It supports file upload, download, on-screen display, namespace operations (move/copy), collection creation and deletion, and locking operations.

Other

Remote

Co-lo

Troubleshooting

Retrieved from "https://wiki.thingsandstuff.org/index.php?title=Server&oldid=31344"