File sharing

From Things and Stuff Wiki
Revision as of 00:33, 28 June 2017 by Milk (talk | contribs) (→‎Services)
Jump to navigation Jump to search


to sort and reorder

See also Network, SSH, Backup (rsync), HTTP, WebDev#WebRTC, Server#WebDAV

General



File systems

Network

  • Bazil is a distributed file system designed for single-person disconnected operation. It lets you share your files across all your computers, with or without cloud services. FUSE is a programming library for writing file systems in userspace, in Go.

Services


File sync

See also SaaS#Storage, Backup

lsyncd

  • lsyncd - designed to synchronize a local directory tree with low profile of expected changes to a remote mirror. Lsyncd is especially useful to sync data from a secure area to a not-so-secure area. Lsyncd watches a local directory trees event monitor interface (inotify or fsevents). It aggregates and combines events for a few seconds and then spawns one (or more) process(es) to synchronize the changes. By default this is rsync. Lsyncd is thus a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or block devices and does not hamper local filesystem performance.

inosync

  • inosync daemon leverages the inotify service available in recent linux kernels to monitor and synchronize changes within directories to remote nodes.

clsync

Syncany

  • Syncany is an open-source cloud storage and filesharing application. It allows users to backup and share certain folders of their workstations using any kind of storage, e.g. FTP, SFTP, WebDAV, Samba, Amazon S3 or Google Storage. While the basic idea is similar to Dropbox, Syncany is open-source and additionally provides data encryption and more flexibility in terms of storage type and provider.

Syncthing

  • Syncthing - Open Source Continuous File Synchronization. Syncthing replaces proprietary sync and cloud services with something open, trustworthy and decentralized. Your data is your data alone and you deserve to choose where it is stored, if it is shared with some third party and how it's transmitted over the Internet. This is the Syncthing project which pursues the following goals: Define a protocol for synchronization of a folder between a number of collaborating devices. This protocol should be well defined, unambiguous, easily understood, free to use, efficient, secure and language neutral. This is called the Block Exchange Protocol. Provide the reference implementation to demonstrate the usability of said protocol. This is the syncthing utility. We hope that alternative, compatible implementations of the protocol will arise. [3]

The discovery mechanism can take a minutes or two to kick in.

  • Port 22000/TCP (or the actual listening port if you have changed the Sync Protocol Listen Address setting.)
  • Port 21027/UDP (for discovery broadcasts on IPv4 and multicasts on IPv6)
vim ~/.config/syncthing/config.xml

Local only access; 

<gui enabled="true" tls="false">
   <address>127.0.0.1:8080</address>
</gui>

No whitelist; 

<gui enabled="true" tls="false">
   <address>0.0.0.0:8080</address>
</gui>

Unison

gtk2 gui is a bit dodge, drops any ssh args given to it.

syncing requires a manual start.

SparkleShare

Seafile

  • Seafile - an open source cloud storage system with features on privacy protection and teamwork. Collections of files are called libraries, and each library can be synced separately. A library can also be encrypted with a user chosen password. Seafile also allows users to create groups and easily sharing files into groups.

git-annex

  • git-annex allows managing files with git, without checking the file contents into git. While that may seem paradoxical, it is useful when dealing with files larger than git can currently easily handle, whether due to limitations in memory, time, or disk space. git-annex is designed for git users who love the command line. For everyone else, the git-annex assistant turns git-annex into an easy to use folder synchroniser.

gut

dvcs-autosync

DirSync Pro

  • DirSync Pro is a small, but powerful utility for file and folder synchronization. DirSync Pro can be used to synchronize the content of one or many folders recursively. Use DirSync Pro to easily synchronize files from your desktop PC to your USB-stick (/Externa HD/PDA/Notebook, ...). Use this USB-stick (/Externa HD/PDA/Notebook, ...) to synchronize files to another desktop PC. Using DirSync Pro you can make incremental backups. In this way you'll spare lots of time because you don't have to copy all the files each time you want to update your backup; only new/modified/larger files would be copied. DirSync Pro provides some preconfigured modes (e.g. Mirror, Backup, Contribute, ?) for common synchronization and backup tasks and some custom modes in which the user could set up the synchronization behavior to meet specific wishes.

Conduit

  • Conduit - a synchronization application for GNOME. It allows you to synchronize your files, photos, emails, contacts, notes, calendar data and any other type of personal information and synchronize that data with another computer, an online service, or even another electronic device. Conduit manages the synchronization and conversion of data into other formats

Csync2

  • Csync2 is a cluster synchronization tool. It can be used to keep files on multiple hosts in a cluster in sync. Csync2 can handle complex setups with much more than just 2 hosts, handle file deletions and can detect conflicts. It is expedient for HA-clusters, HPC-clusters, COWs and server farms.

Bittorrent Sync

Ubuntu One

Other

  • Pydio is an open source software that turns instantly any server (on premise, NAS, cloud IaaS or PaaS) into a file sharing platform for your company. It is an alternative to SaaS Boxes and Drives, with more control, safety and privacy, and favorable TCOs.

https://github.com/pydio

  • bitpocket is a small but smart script that does 2-way directory synchronization. It uses rsync to do efficient data transfer and tracks local file creation/removal to avoid known rsync problem when doing 2-way syncing with deletion. bitpocket can use any server which you have ssh access to for its central storage. If you have gigabytes of free disk space on your hosting server you can finally make use of it.



Client-server

See Network#FTP

DCC

Napster

Direct Connect

  • NCurses Direct Connect - Ncdc is a modern and lightweight direct connect client with a friendly ncurses interface.

Cloud

See Hosting#Storage

  • Volafile.io aims to provide a simple real-time filesharing service. Users share files in so called 'rooms', which each consist of a chat and a list of uploaded files. The list of files automatically shows new files as they are uploaded. [5]



Direct

Demosaw

YouTransfer

  • YouTransfer - a simple but elegant self-hosted file transfer & sharing solution. It is an alternative to paid services like Dropbox and WeTransfer by offering similar features but without limitations, price plans and a lengthy privacy policy. You remain in control of your files. Created to be installed behind the firewall on private servers, YouTransfer aims to empower organisations and individuals that wish to combine easy-to-use file transfer tooling with security and control. [8]

P2P

See also Encrypted#Darknet



DHT

  • P-Grid is a truly decentralized structured P2P system which does not require central coordination or knowledge. It is based purely on randomized algorithms and local interactions and targeted at environments with low online probabilities of peers. P-Grid differs from other approaches such as Chord, CAN, Pastry, etc. in terms of practical applicability (especially in respect to dynamic network environments), algorithmic foundations (randomized algorithms with probabilistic guarantees), robustness, and flexibility.
  • TomP2P is an advanced DHT, which stores multiple values for a key. Each peer has a table (either disk-based or memory-based) to store its values. A single value can be queried / updated with a secondary key. The underlying communication framework uses Java NIO to handle many concurrent connections.
  • Tonika is an administration-free platform for large-scale open-membership (social) networks with robust security, anonymity, resilience and performance guarantees.
  • SIPDHT project is to provide a library to be used in applications for creating and using SIP based distributed hash tables. The algorithm implemented is loosely based on P2P SIP work (draft-bryan-sipping-p2p-02) and its primary intent is to be a work bench for the specification process.
  • http://sipdht.sourceforge.net/sipdht2/
  • sinchana is a scalable and stable DHT implementation written using Apache Thrift. The communication protocol of the DHT is implemented using Thrift. The advantage of Thrift service protocol is it's small message size and the less usage of CPU.

File networks

  • Forban - a simple link-local opportunistic p2p free software (or how to share files with your local neighbors)

Gnutella

FastTrack

eDonkey network

Soulseek

2006

  • https://en.wikipedia.org/wiki/Soulseek - a peer-to-peer (P2P) file-sharing network and application. The term Soulseek might refer to (1) one of the two networks, or (2) one of the three official user client interfaces. Soulseek is used mostly to exchange music, although users are able to share a variety of files.



Overnet

Perfect Dark

Advanced Direct Connect

Multi-network Software

giFT
  • giFT is a collection of various software components geared towards improving the overall usability of a multitude of peer-to-peer file-sharing networks. This goal is accomplished by unifying the components such that a user of the software may choose any of the available interfaces to access any of the available networks. The central application, giftd, uses a plugin-based architecture capable of using multiple networks simultaneously through a single user interface.
MLDonkey
P2P-GUI
  • P2P-GUI is a remote web interface for MLdonkey, aMule, rTorrent, Transmission and giFT. This GUI tries to combine different GUIs for p2p programs in one application/project to save overall work and to cover many different needs and tastes.

BitTorrent

Libraries

Servers

  • XBT is high-performance low-overhead BitTorrent software.
  • Gazelle is a web framework geared towards private BitTorrent trackers. Although naturally focusing on music, it can be modified for most needs. Gazelle is written in PHP, JavaScript, and MySQL.

Clients

rTorrent

Extensions

Trackers

  • OpenBitTorrent is a bittorrent tracker free for anyone to use. You don't need to register, upload or index a torrent anywhere, all you have to do is to include the OpenBitTorrent tracker URL in your torrent.

Search

Seedbox

Repos

Tools


Streaming

BT Live


to sort






WebTorrent
  • WebTorrent - a streaming torrent client for node.js and the browser. YEP, THAT'S RIGHT. THE BROWSER. It's written completely in JavaScript – the language of the web – so the same code works in both runtimes. In node.js, this module is a simple torrent client, using TCP and UDP to talk to other torrent clients. In the browser, WebTorrent uses WebRTC (data channels) for peer-to-peer transport. It can be used without browser plugins, extensions, or installations. It's Just JavaScript™. [18] [19]



Beaker

WebRTC

  • https://en.wikipedia.org/wiki/WebRTC - an API definition drafted by the World Wide Web Consortium (W3C) that supports browser-to-browser applications for voice calling, video chat, and P2P file sharing without the need of either internal or external plugins.

Two peers first need to find one another. This signalling step is purposefully omitted from the WebRTC specification, since the WebRTC protocol is not specific to browsers and can be used by any number of different devices in different circumstances. Sometimes this is hindered by network topology realities, e.g. non-permissive firewalls or NAT routers. In this case it is physically impossible for the two peers to communicate in a P2P manner and a third party relay is required; this is included in the WebRTC specification in the form of a TURN server. [22]



  • https://code.google.com/p/webrtc4all/ a WebRTC extension for Safari, Firefox, Opera and IE9+. This extension is part of sipML5 solution and implements Javascript PeerConnection API as defined in draft-uberti-rtcweb-jsep-02.
  • webrtc2sip is a smart and powerful gateway using RTCWeb and SIP to turn your browser into a phone with audio, video and SMS capabilities. The gateway allows your web browser to make and receive calls from/to any SIP-legacy network or PSTN. As an example, you will be able to make a call from your preferred web browser to a SIP-legacy softphone (e.g. xlite) or mobile/fixed phone. The gateway contains four modules: SIP Proxy | RTCWeb Breaker | Media Coder | Click-to-Call.
  • sipml5 - The world's first HTML5 SIP client (WebRTC). Source code freely provided to you by http://www.doubango.org'> Doubango Telecom ® under BSD terms. This is the world's first open source HTML5 SIP client (May 12, 2012) entirely written in javascript for integration in social networks (FaceBook, Twitter, Google+), online games, e-commerce sites... No extension, plugin or gateway is needed. The media stack rely on WebRTC. The client can be used to connect to any SIP or IMS network from your preferred browser to make and receive audio/video calls and instant messages. It's also possible to make calls to PSTN or any SIP-legacy network using webrtc2sip.org. The protocol parsers (SIP, SDP...) are highly optimized using http://www.complang.org/ragel/'>Ragel lookup tables and is suitable for embedded systems with limited memory and low computing power.



  • Janus - a WebRTC Gateway developed by Meetecho conceived to be a general purpose one. As such, it doesn't provide any functionality per se other than implementing the means to set up a WebRTC media communication with a browser, exchanging JSON messages with it, and relaying RTP/RTCP and messages between browsers and the server-side application logic they're attached to. Any specific feature/application is provided by server side plugins, that browsers can then contact via the gateway to take advantage of the functionality they provide. Example of such plugins can be implementations of applications like echo tests, conference bridges, media recorders, SIP gateways and the like.


  • Kurento - a WebRTC media server and a set of client APIs making simple the development of advanced video applications for WWW and smartphone platforms. Kurento Media Server features include group communications, transcoding, recording, mixing, broadcasting and routing of audiovisual flows. As a differential feature, Kurento Media Server also provides advanced media processing capabilities involving computer vision, video indexing, augmented reality and speech analysis. Kurento modular architecture makes simple the integration of third party media processing algorithms (i.e. speech recognition, sentiment analysis, face recognition, etc.), which can be transparently used by application developers as the rest of Kurento built-in features.


Services


  • Baron is a WebRTC-based web application that allows people to have peer-to-peer text, audio and video conversations with permanent accounts and contact lists. The back-end is written in JavaScript (Node.js), the signaling channel uses Socket.io. The front-end is written in Ember.js. Authentication is done using Mozilla Persona.



  • P is a small framework used to create browser-to-browser networks (as opposed to just a connection).




  • Matrix is an open standard for decentralised communication, providing simple HTTP APIs and open source reference implementations for securely distributing and persisting JSON over an open federation of servers. Fully distributed persistent chatrooms with no single points of control or failure. WebRTC signalling transport for interoperable VoIP and video calling. Exchanging and persisting data between devices and services.



Encrypted / anon

WASTE

  • WASTE is an anonymous, secure, and encryped collaboration tool which allows users to both share ideas through the chat interface and share data through the download system.

The suggested size for a WASTE network (referred to as a "mesh" by users) is 10-50 nodes, though it has been suggested that the size of the network is less critical than the ratio of nodes willing to route traffic to those that are not.

Freenet

  • Freenet - Share files, chat on forums, browse and publish, anonymously and without fear of blocking or censorship! While users can insert data into the network, there is no way to delete data. Due to the anonymity, no node knows who is the 'owner' of a piece of data. The only way data can be removed is if users don't request the data.

RetroShare

  • RetroShare is a Open Source cross-platform, Friend-2-Friend and secure decentralised communication platform. It lets you to securely chat and share files with your friends and family, using a web-of-trust to authenticate peers and OpenSSL to encrypt all communication. RetroShare provides filesharing, chat, messages, forums and channels

MUTE

OneSwarm

OFF

  • OFF is a highly connected peer-to-peer distributed file system. The unique feature of this system is that it stores all of its internal data as meaningless multi-use data blocks. In other words there is not a one to one mapping between a stored block and its use in a accessed file. Each stored block is simultaneously used to access many different files. Individually however, each block is nothing but arbitrary digital white noise.

Free Haven Project

  • Free Haven Project aims to deploy a system for distributed, anonymous, persistent data storage which is robust against attempts by powerful adversaries to find and destroy any stored data. This model of decentralized system has been classified as peer-to-peer by recent popular media.

Tahoe-LAFS

  • Tahoe-LAFS is a Free and Open cloud storage system. It distributes your data across multiple servers. Even if some of the servers fail or are taken over by an attacker, the entire filesystem continues to function correctly, preserving your privacy and security.

Osiris

Ori

MaidSafe


Other

  • Cryptomator - Free client-side AES encryption for your cloud files. Open source software: No backdoors, no registration.

Keybase

ZeroNet

Downloading


Magnet links

Physical

Retrieved from "https://wiki.thingsandstuff.org/index.php?title=File_sharing&oldid=41935"