Stack

From Things and Stuff Wiki
Revision as of 09:33, 10 November 2017 by Milk (talk | contribs) (→‎Other)
Jump to navigation Jump to search


See also Server, Box, Distros, E-mail#Server, Platforms, BOA

to sort out big time

General

Hardware

Open Compute

Clustering

Orchestration and management

See also Hosting#PaaS

OpenStack

  • Nova is the project name for OpenStack Compute, a cloud computing fabric controller, the main part of an IaaS system. Individuals and organizations can use Nova to host and manage their own cloud computing systems. Nova originated as a project out of NASA Ames Research Laboratory.
  • Swift is a highly available, distributed, eventually consistent object/blob store. Organizations can use Swift to store lots of data efficiently, safely, and cheaply.
  • Glance provides services for discovering, registering, and retrieving virtual machine images. Glance has a RESTful API that allows querying of VM image metadata as well as retrieval of the actual image.
  • Cinder provides an infrastructure for managing volumes in OpenStack. It was originally a Nova component called nova-volume, but has become an independent project since the Folsom release.
  • Neutron (was Quantum) provides "networking as a service" between interface devices (e.g., vNICs) managed by other Openstack services (e.g., nova).
  • Keystone is the identity service used by OpenStack for authentication (authN) and high-level authorization (authZ). It currently supports token-based authN and user-service authorization. It has recently been rearchitected to allow for expansion to support proxying external services and AuthN/AuthZ mechanisms such as oAuth, SAML and openID in future versions.

OpenNebula

  • OpenNebula.org is an open-source project developing the industry standard solution for building and managing virtualized enterprise data centers and enterprise private clouds.
    • OpenNebula is an open-source cloud computing toolkit for managing heterogeneous distributed data center infrastructures.

OpenShift

Cloud Foundry

Juju

  • Juju enables you to use Charms to deploy your application architectures to EC2, OpenStack, Azure your data center and even your own Ubuntu based laptop. Moving between environments is simple giving you the flexibility to switch hosts whenever you want — for free.

Landscape

Eucalyptus

Cocaine

  • Cocaine (Configurable Omnipotent Custom Applications Integrated Network Engine) is an open-source PaaS (platform as a service) system for creating custom cloud hosting apps that are similar to Google App Engine or Heroku. Any library or service can be implemented as a service in Cocaine using a special API. Several indispensable services have already been implemented this way, including a service for detecting a user's region or language, a service for accessing MongoDB storage, and a URL fetcher.

MaestroNG

  • MaestroNG - command-line utility that allows for automatically managing the orchestrated deployment and bring up of a set of service instance containers that compose an environment on a set of target host machines. Each host machine is expected to run a Docker daemon. Maestro will then contact the Docker daemon of each host in the environment to figure out the status of the environment and what actions to take based on the requested command.

Dokku

  • Dokku - Docker powered mini-Heroku. The smallest PaaS implementation you've ever seen.

Project Atomic

  • Project Atomic integrates the tools and patterns of container-based application and service deployment with trusted operating system platforms to deliver an end-to-end hosting architecture that's modern, reliable and secure. [3]

Deis

Phusion Passenger

  • Phusion Passenger - Web server and application server. Designed to be fast, robust and lightweight. Makes web app deployments a lot simpler and less complex, by managing your apps' processes and resources for you.

Consul

  • Consul is a tool for service discovery and configuration. Consul is distributed, highly available, and extremely scalable.

GNU parallel

  • GNU parallel is a shell tool for executing jobs in parallel using one or more computers. A job can be a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. A job can also be a command that reads from a pipe. GNU parallel can then split the input and pipe it into commands in parallel.

Skydock

  • Skydock monitors docker events when containers start, stop, die, kill, etc and inserts records into a dynamic DNS server skydns. This allows standard DNS queries for services running inside docker containers. Because lets face it, if you have to modify your application code to work with other service discovery solutions you might as well just give up. DNS just works and it works well.

Flynn

  • Flynn simplifies deploying and maintaining applications. Instead of using complex configuration management systems, Flynn allows self-serve management of containerized deployments, making life easier for ops and developers. [5] [6]

Gaudi

Terraform

Nixos

VirtKick

Dokku

DC/OS

Service discovery

See also Network#Distributed

to merge into Configuration and management?

Zookeeper

  • ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. All of these kinds of services are used in some form or another by distributed applications. Each time they are implemented there is a lot of work that goes into fixing the bugs and race conditions that are inevitable. Because of the difficulty of implementing these kinds of services, applications initially usually skimp on them, which make them brittle in the presence of change and difficult to manage. Even when done correctly, different implementations of these services lead to management complexity when the applications are deployed.

Doozer

  • Doozer is a highly-available, completely consistent store for small amounts of extremely important data. When the data changes, it can notify connected clients immediately (no polling), making it ideal for infrequently-updated data for which clients want real-time updates. Doozer is good for name service, database master elections, and configuration data shared between several machines. See When Should I Use It?, below, for details.

etcd

  • etcd - A highly-available key value store for shared configuration and service discovery. etcd is inspired by Apache ZooKeeper and doozer, with a focus on being: Simple: curl'able user facing API (HTTP+JSON), Secure: optional SSL client cert authentication, Fast: benchmarked 1000s of writes/s per instance, Reliable: properly distributed using Raft

SkyDNS

  • SkyDNS is a distributed service for announcement and discovery of services. It leverages Raft for high-availability and consensus, and utilizes DNS queries to discover available services. This is done by leveraging SRV records in DNS, with special meaning given to subdomains, priorities and weights.

Serf

  • Serf is a decentralized solution for service discovery and orchestration that is lightweight, highly available, and fault tolerant.

Used by Flynn.

Discoverd

  • Discoverd - A simple but powerful service discovery system written in Go. It's currently backed by etcd, but can be extended to use ZooKeeper or other distributed consistent stores.

Provision and configuration

See also *nix#Management

Puppet

apt-get install puppet puppetmaster facter rubygems puppet-module

Foreman

Boxen

Chef


Salt

  • Salt is a configuration management system, capable of maintaining remote nodes in defined states (for example, ensuring that specific packages are installed and specific services are running), a distributed remote execution system used to execute commands and query data on remote nodes, either individually or by arbitrary selection criteria

Ansible

  • Sovereign – Ansible playbooks to build and maintain your own personal cloud [11]

Foreman

  • The Foreman is a complete lifecycle management tool for physical and virtual servers. Through deep integration with configuration management, DHCP, DNS, TFTP, and PXE-based unattended installations, Foreman manages every stage of the lifecycle of your physical or virtual servers. The Foreman provides comprehensive, auditable interaction facilities including a web frontend and robust, RESTful API.

Fabric

Capistrano

  • Capistrano is a utility and framework for executing commands in parallel on multiple remote machines, via SSH. It uses a simple DSL (borrowed in part from Rake) that allows you to define tasks, which may be applied to machines in certain roles. It also supports tunneling connections via some gateway machine to allow operations to be performed behind VPN's and firewalls. Capistrano was originally designed to simplify and automate deployment of web applications to distributed environments, and originally came bundled with a set of tasks designed for deploying Rails applications.

CFEngine

Synapse

  • Synapse enables you to remotely manage a large number of hosts. It brings together features of Configuration Management and Orchestration in a lightweight framework. Written in Python and using AMQP for messaging between the nodes.

Archipel

Ubuntu Orchestra

SM Framework

  • SM framework is a framework for developing, maintaining and sharing scripts, while exposing them for easy automation via a clean, consistent DSL-centric command-line interface. SM Framework Core (sm) is installed onto a system and is then used to manage that system through extensions. The primary requirement for sm is zsh (>= 4.2), secondarily are standard unix tools such as {bash (>= 3.2), GNU awk, GNU sed, grep, which, ls, cp, tar, curl, gunzip, bunzip2}. While not a direct dependency, nearly all open source extensions currently install via git (> 1.7).

Other

  • Bcfg2 - helps system administrators produce a consistent, reproducible, and verifiable description of their environment, and offers visualization and reporting tools to aid in day-to-day administrative tasks. It is the fifth generation of configuration management tools developed in the Mathematics and Computer Science Division of Argonne National Laboratory. It is based on an operational model in which the specification can be used to validate and optionally change the state of clients, but in a feature unique to Bcfg2 the client's response to the specification can also be used to assess the completeness of the specification. Using this feature, Bcfg2 provides an objective measure of how good a job an administrator has done in specifying the configuration of client systems. Bcfg2 is therefore built to help administrators construct an accurate, comprehensive specification. Bcfg2 has been designed from the ground up to support gentle reconciliation between the specification and current client states. It is designed to gracefully cope with manual system modifications. Finally, due to the rapid pace of updates on modern networks, client systems are constantly changing; if required in your environment, Bcfg2 can enable the construction of complex change management and deployment strategies.


  • Fossil - a simple, high-reliability, distributed software configuration management system with these advanced features: In addition to doing distributed version control like Git and Mercurial, Fossil also supports bug tracking, wiki, and technotes. Fossil has a built-in and intuitive web interface with a rich variety of information pages (examples) promoting situational awareness. Fossil is a single self-contained stand-alone executable. To install, simply download a precompiled binary for Linux, Mac, OpenBSD, or Windows and put it on your $PATH. Easy-to-compile source code is also available. No custom protocols or TCP ports. Fossil uses ordinary HTTP (or HTTPS or SSH) for network communications, so it works fine from behind restrictive firewalls, including proxies. The protocol is bandwidth efficient to the point that Fossil can be used comfortably over dial-up or over the exceedingly slow Wifi on airliners. No server is required, but if you want to set one up, Fossil supports four easy server configurations. Fossil supports "autosync" mode which helps to keep projects moving forward by reducing the amount of needless forking and merging often associated with distributed projects. Fossil stores content using an enduring file format in an SQLite database so that transactions are atomic even if interrupted by a power loss or system crash. Automatic self-checks verify that all aspects of the repository are consistent prior to each commit. Fossil works with repository files (a database with the project's complete history) and with checked-out local trees (the working directory you use to do your work). (See definitions for more background.) The workflow looks like this: Create or clone a repository file. (fossil init or fossil clone) Check out a local tree. (fossil open) Perform operations on the repository (including repository configuration).


  • cdist - a usable configuration management system. It adheres to the KISS principle and is being used in small up to enterprise grade environments. cdist is an alternative to other configuration management systems like bcfg2, chef, cfengine and puppet.



  • Augeas is a configuration editing tool. It parses configuration files in their native formats and transforms them into a tree. Configuration changes are made by manipulating this tree and saving it back into native config files.
  • Itamae - Configuration management tool inspired by Chef, but simpler and lightweight. Formerly known as Lightchef.
  • Holo users ship configuration in packages, usually called "holograms". These can be built with the regular package building tools (debuild, rpmbuild, makepkg, etc.) or with Holo's own holo-build tool that offers a much more pleasant syntax and process. Let's go through an example hologram that installs and starts an OpenSSH server and tweaks some of its configuration.


  • FAI is a non-interactive system to install, customize and manage Linux systems and software configurations on computers as well as virtual machines and chroot environments, from small networks to large-scale infrastructures like clusters and cloud environments. It's a tool for unattended mass deployment of Linux. You can take one or more virgin PC's, turn on the power, and after a few minutes, the systems are installed, and completely configured to your exact needs, without any interaction necessary.

To sort

  • Centmin Mod is for installation on CentOS only and written by George Liu (eva2000) with the addition of a shell menu based installer (shown on the right). Centmin Mod shell based menu allows you to do basic Nginx & PHP related management including upgrading or downgrading Nginx & PHP or setting up Nginx vhosts.


  • Ajenti includes dozens of pre-made plugins that let you configure both your OS and server software. List of supported software includes Apache, BIND9, Cron, CTDB, DHCPD, NFSD, Iptables, Munin, MySQL, Netatalk, NGINX, PostgreSQL, Samba, lm-sensors, Squid 3, Supervisor.
  • Apache Mesos is a cluster manager that provides efficient resource isolation and sharing across distributed applications, or frameworks. It can run Hadoop, MPI, Hypertable, Spark, and other applications on a dynamically shared pool of nodes.
  • https://github.com/mesosphere/marathon Mesos framework for long running services



  • Symbiosis comes as standard with all new Bytemark servers, so if you're a recent customer or have re-imaged your system recently, it's ready to go. You can easily host PHP, Perl/CGI, htaccess files and MySQL, including any custom modules and settings that you might need. You have root access, so you don't need to ask permission to host new domains, different PHP settings, or create a database. You can even host any other Linux technology such as Java, Rails or Erlang. you can host any number of mailboxes under a domain, including forwarders, IMAP/POP3 boxes and autoresponders. There is a simple local anti-spam service based on Spamassassin, but if you need managed filtering you can quickly elect to pass your email through our anti-spam service.

Tasksel

TurnKey Linux

Retrieved from "https://wiki.thingsandstuff.org/index.php?title=Stack&oldid=42569"