HTTP

From Things and Stuff Wiki
Revision as of 00:11, 15 April 2013 by Milk (talk | contribs) (→‎Proxy)
Jump to navigation Jump to search


Basics

  • HTTPie is a CLI, cURL-like tool for humans

User and group

See also *nix#Users

Servers

python2 

 python -m SimpleHTTPServer 8000

python3 

python -m http.server 5674
  • h5ai makes browsing directories on HTTP web servers more pleasant. Directory listings get styled in a modern way and browsing through the directories is enhanced by different views, a breadcrumb and a tree overview.

Nginx

  • nginx [engine x] is a free, open-source, high-performance HTTP server and reverse proxy, as well as an IMAP/POP3 proxy server. Igor Sysoev started development of Nginx in 2002, with the first public release in 2004. Nginx now hosts nearly 12.18% (22.2M) of active sites across all domains. Nginx is known for its high performance, stability, rich feature set, simple configuration, and low resource consumption.

Guides

Compression

gzip_types text/plain text/css application/x-javascript text/xml application/xml application/xml+rss
text/javascript application/json image/svg+xml application/vnd.ms-fontobject application/x-font-ttf
font/opentype;

Webfonts

Nginx uses a file’s mime type declaration to decide whether or not to apply compression to that file, and so we must first ensure that the four types of web font files have mime types configured.

/etc/nginx/mime.types 

application/vnd.ms-fontobject      eot;
application/x-font-ttf             ttf;
font/opentype                      ott;
font/x-woff                        woff;

remove; 

application/octet-stream          eot;
application/vnd.oasis.opendocument.text-template  ott;

Configuration

location / { 
   auth_basic            "Restricted";
   auth_basic_user_file  /etc/nginx/conf.d/htpasswd;
 }

Listening on specific IP will override a wildcard IP catch-all.

Site setup

Site folder location can vary. 

/var/www
  debian
/etc/nginx/html
  arch linux

Create 'Server Block' (vhost) config file in 

/etc/nginx/sites-available

and later ln -s them in 

/etc/nginx/sites-enabled

Enable logging in vhost conf; 

error_log /var/log/nginx-vhostnamehere.log error;

SSL

Make sure wildcard SSL is in http stanza and that any specific server is listening on 443.

Modules

  • Nginx modules must be selected during compile, run-time selection of modules is not currently supported.
  • HttpLimitConnModule - This module makes it possible to limit the number of concurrent connections for a defined key such as, for example, an ip address.
printf "John:$(openssl passwd -crypt V3Ry)\n" >> .htpasswd # this example uses crypt encryption
printf "Mary:$(openssl passwd -apr1 SEcRe7)\n" >> .htpasswd # this example uses apr1 (Apache MD5) encryption
printf "Jane:$(openssl passwd -1 V3RySEcRe7)\n" >> .htpasswd # this example uses MD5 encryption
(PWD="SEcRe7PwD";SALT="$(openssl rand -base64 3)";SHA1=$(printf "$PWD$SALT" | openssl dgst -binary -sha1 | \ sed 's#$#'"$SALT"'#' | base64);printf "Jim:{SSHA}$SHA1\n" >> .htpasswd) # this example uses SSHA encryptio

Tools

Directory listing

autoindex on;

Lua

Forks

Apache

  • The 5G Blacklist helps reduce the number of malicious URL requests that hit your website. It’s one of many ways to improve the security of your site and protect against evil exploits, bad requests, and other nefarious garbage.

.htaccess

lighttpd

Other

CORS

Proxy

Testing

  • RED is a robot that checks HTTP resources to see how they'll behave, pointing out common problems and suggesting improvements. Although it is not a HTTP conformance tester, it can find a number of HTTP-related issues.
  • Browser SOA Debugger - Depending on the view of things this is just an enhanced HTTP output formatter for tcpdump streams, or the ultimate debugger for complex HTTP oriented SOA architectures which visualizes the full HTTP interactions in a readable, reproducible way so that you can see what is actually going on in your backend.

Performance

See also Server#Performance

Guides

Cookieless

Cookie free domain for static content so cooke isn't sent with request. Root domain cookies apply to all subdomain cookies, though using www. (ugh!) works. Use another domain A record to point to the site.(?)

ETags

Caching

Varnish

Load balancing

CDN

  • CoralCDN is a decentralized, self-organizing, peer-to-peer web-content distribution network. CoralCDN leverages the aggregate bandwidth of volunteers running the software to absorb and dissipate most of the traffic for web sites using the system.
.nyud.net

p2p;

Testing

  • etc.
  • GTmetrix uses Google Page Speed and Yahoo! YSlow to grade your site's performance and provides actionable recommendations to fix these issues.
  • Blitz does cloud based load and performance testing ising Sinatra, Rails and node.js.
    • Free: Sprint all you want, Rush all you want, 250 concurrent users, 1 minute rushes
  • Engulf is a scalable, distributed HTTP benchmarker, designed to let you spin up and coordinate a cluster of workers with nothing more than a single JAR. Engulf's backend is written in clojure, the frontend in javascript.

Web server logs

Combined Log Format 

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" combined

old skool;

Apache

Nginx

error_log  /var/log/nginx/domain.name/error.log;
access_log  /var/log/nginx/domain.name/access.log;

GoAccess

  • GoAccess is an open source real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems. It provides fast and valuable HTTP statistics for system administrators that require a visual server report on the fly.

Control panels

Analytics

Referer

Piwik

  • Piwik is downloadable, Free/Libre (GPLv3 licensed) real time web analytics software. It provides you with detailed reports on your website visitors; the search engines and keywords they used, the language they speak, your popular pages, and much more.

WordPress

Clicky

Google

other

Upload

Quick screipts for uploading;

404

Other

HTTP/2.0

see SPDY and such conversation

Retrieved from "https://wiki.thingsandstuff.org/index.php?title=HTTP&oldid=10333"