|
Grid File System
A grid file system is a computer file system whose goal is improved reliability and availability by taking advantage of many smaller file storage areas. Components File systems contain up to three components: * File table (FAT table, MFT, etc.) * File data * Metadata (user permissions, etc.) A grid file system would have similar needs: * File table (or search index) * File data * Metadata Comparisons Because file systems are designed to appear as a single disk for a single computer to manage (entirely), many new challenges arise in a grid scenario whereby any single disk within the grid should be capable of handling requests for any data contained in the grid. Features Most file storage utilizes layers of redundancy to achieve a high level of data protection (inability to lose data). Current means of redundancy include replication and parity checks. Such redundancy can be implemented via a RAID array (whereby multiple physical disks appear to a local computer as a single disk, w ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Metadata
Metadata (or metainformation) is "data that provides information about other data", but not the content of the data itself, such as the text of a message or the image itself. There are many distinct types of metadata, including: * Descriptive metadata – the descriptive information about a resource. It is used for discovery and identification. It includes elements such as title, abstract, author, and keywords. * Structural metadata – metadata about containers of data and indicates how compound objects are put together, for example, how pages are ordered to form chapters. It describes the types, versions, relationships, and other characteristics of digital materials. * Administrative metadata – the information to help manage a resource, like resource type, and permissions, and when and how it was created. * Reference metadata – the information about the contents and quality of Statistical data type, statistical data. * Statistical metadata – also called process data, may ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Tahoe-LAFS
Tahoe-LAFS (Tahoe Least-Authority File Store) is a free and open, secure, decentralized, fault-tolerant, distributed data store and distributed file system. It can be used as an online backup system, or to serve as a file or Web host similar to Freenet, depending on the front-end used to insert and access files in the Tahoe system. Tahoe can also be used in a RAID-like fashion using multiple disks to make a single large Redundant Array of Inexpensive Nodes (RAIN) pool of reliable data storage. The system is designed and implemented around the " principle of least authority" (POLA), described by Brian Warner (one of the project's original founders) as the idea "that any component of the system should have as little power of authority as it needs to get its job done". Strict adherence to this convention is enabled by the use of cryptographic capabilities that provide the minimum set of privileges necessary to perform a given task by asking agents. A RAIN array acts as a stora ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Clustered File System
A clustered file system (CFS) is a file system which is shared by being simultaneously mounted on multiple servers. There are several approaches to clustering, most of which do not employ a clustered file system (only direct attached storage for each node). Clustered file systems can provide features like location-independent addressing and redundancy which improve reliability or reduce the complexity of the other parts of the cluster. Parallel file systems are a type of clustered file system that spread data across multiple storage nodes, usually for redundancy or performance. Shared-disk file system A shared-disk file system uses a storage area network (SAN) to allow multiple computers to gain direct disk access at the block level. Access control and translation from file-level operations that applications use to block-level operations used by the SAN must take place on the client node. The most common type of clustered file system, the shared-disk file systemby addi ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Cooperative Storage Cloud
A cooperative storage cloud is a decentralized model of networked online storage where data is stored on multiple computers ( nodes), hosted by the participants cooperating in the cloud. For the cooperative scheme to be viable, the total storage contributed in aggregate must be at least equal to the amount of storage needed by end users. However, some nodes may contribute less storage and some may contribute more. There may be reward models to compensate the nodes contributing more. Unlike a traditional storage cloud, a cooperative does not directly employ dedicated servers for the actual storage of the data, thereby eliminating the need for a significant dedicated hardware investment. Each node in the cooperative runs specialized software which communicates with a centralized control and orchestration server, thereby allowing the node to both consume and contribute storage space to the cloud. The centralized control and orchestration server requires several orders of magnitud ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Distributed Hash Table
A distributed hash table (DHT) is a Distributed computing, distributed system that provides a lookup service similar to a hash table. Key–value pairs are stored in a DHT, and any participating node (networking), node can efficiently retrieve the value associated with a given key. The main advantage of a DHT is that nodes can be added or removed with minimum work around re-distributing keys. ''Keys'' are unique identifiers which map to particular ''values'', which in turn can be anything from addresses, to Electronic document, documents, to arbitrary Data (computing), data. Responsibility for maintaining the mapping from keys to values is distributed among the nodes, in such a way that a change in the set of participants causes a minimal amount of disruption. This allows a DHT to scale (computing), scale to extremely large numbers of nodes and to handle continual node arrivals, departures, and failures. DHTs form an infrastructure that can be used to build more complex services, su ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
GlusterFS
Gluster Inc. (formerly known as Z RESEARCH) was a software company that provided an open source platform for scale-out public and private cloud storage. The company was privately funded and headquartered in Sunnyvale, California, with an engineering center in Bangalore, India. Gluster was funded by Nexus Venture Partners and Index Ventures. Gluster was acquired by Red Hat on October 7, 2011. History The name ''Gluster'' comes from the combination of the terms ''GNU'' and ''cluster''. Despite the similarity in names, Gluster is not related to the Lustre file system and does not incorporate any Lustre code. Gluster based its product on ''GlusterFS'', an open-source software-based network-attached filesystem that deploys on commodity hardware. The initial version of GlusterFS was written by Anand Babu Periasamy, Gluster's founder and CTO. In May 2010 Ben Golub became the president and chief executive officer. Red Hat became the primary author and maintainer of the GlusterFS op ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
GNUnet
GNUnet is a software framework for decentralized, peer-to-peer networking and an official GNU package. The framework offers link encryption, peer discovery, resource allocation, communication over many transports (such as TCP, UDP, HTTP, HTTPS, WLAN and Bluetooth) and various basic peer-to-peer algorithms for routing, multicast and network size estimation. GNUnet's basic network topology is that of a mesh network. GNUnet includes a distributed hash table (DHT) which is a randomized variant of Kademlia that can still efficiently route in small-world networks. GNUnet offers a " F2F topology" option for restricting connections to only the users' trusted friends. The users' friends' own friends (and so on) can then indirectly exchange files with the users' computer, never using its IP address directly. GNUnet uses Uniform resource identifiers (not approved by IANA, although an application has been made). GNUnet URIs consist of two major parts: the module and the module spe ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Grid Computing
Grid computing is the use of widely distributed computer resources to reach a common goal. A computing grid can be thought of as a distributed system with non-interactive workloads that involve many files. Grid computing is distinguished from conventional high-performance computing systems such as cluster computing in that grid computers have each node set to perform a different task/application. Grid computers also tend to be more heterogeneous and geographically dispersed (thus not physically coupled) than cluster computers. Although a single grid can be dedicated to a particular application, commonly a grid is used for a variety of purposes. Grids are often constructed with general-purpose grid middleware software libraries. Grid sizes can be quite large. Grids are a form of distributed computing composed of many networked loosely coupled computers acting together to perform large tasks. For certain applications, distributed or grid computing can be seen as a special ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Grid-oriented Storage
Grid-oriented Storage (GOS) was a term used for data storage by a university project during the era when the term grid computing was popular. Description GOS was a successor of the term network-attached storage (NAS). GOS systems contained hard disks, often RAIDs (redundant arrays of independent disks), like traditional file servers. GOS was designed to deal with long-distance, cross-domain and single-image file operations, which is typical in Grid environments. GOS behaves like a file server via the file-based GOS-FS protocol to any entity on the grid. Similar to GridFTP, GOS-FS integrates a parallel stream engine and Grid Security Infrastructure (GSI). Conforming to the universal VFS (Virtual Filesystem Switch), GOS-FS can be pervasively used as an underlying platform to best utilize the increased transfer bandwidth and accelerate the NFS/CIFS-based applications. GOS can also run over SCSI, Fibre Channel or iSCSI, which does not affect the acceleration performance, offerin ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
InterPlanetary File System
The InterPlanetary File System (IPFS) is a protocol, hypermedia and file sharing peer-to-peer network for sharing data using a distributed hash table to store provider information. By using content addressing, IPFS uniquely identifies each file in a global namespace that connects IPFS hosts, creating a distributed system of file storage and sharing. IPFS allows users to host and receive content in a manner similar to BitTorrent. As opposed to a centrally located server, IPFS is built around a decentralized system of user-operators who hold a portion of the overall data. Any user in the network can serve a file by its content address, and other peers in the network can find and request that content from any node who has it using a distributed hash table (DHT). In contrast to traditional location-based protocols like HTTP and HTTPS, IPFS uses content-based addressing to provide a decentralized alternative for distributing the World Wide Web. Design The InterPlanetary File ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
List Of Volunteer Computing Projects
This is a comprehensive list of volunteer computing projects, which are a type of distributed computing where volunteers donate computing time to specific causes. The donated computing power comes from idle CPUs and GPUs in personal computers, video game consoles, and Android (operating system), Android devices. Each project seeks to utilize the computing power of many internet connected devices to solve problems and perform tedious, repetitive research in a very cost effective manner. Active projects Completed projects See also * List of grid computing projects * List of citizen science projects * List of crowdsourcing projects * List of free and open-source Android applications * Berkeley Open Infrastructure for Network Computing#Projects, List of Berkeley Open Infrastructure for Network Computing (BOINC) projects References {{DEFAULTSORT:Volunteer computing projects Volunteer computing Science in society Science-related lists Computing-related lists Lists of ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
XtreemFS
XtreemFS is an object-based, distributed file system for wide area networks.F. Hupfeld, T. Cortes, B. Kolbeck, E. Focht, M. Hess, J. Malo, J. Marti, J. Stender, E. Cesario"XtreemFS - a case for object-based storage in Grid data management" VLDB Workshop on Data Management in Grids. In: Proceedings of 33rd International Conference on Very Large Data Bases (VLDB) Workshops, 2007. XtreemFS' outstanding feature is full (all components) and real (all failure scenarios, including network partitions) fault tolerance, while maintaining POSIX file system semantics. Fault-tolerance is achieved by using Paxos-based lease negotiation algorithms and is used to replicate files and metadata. SSL and X.509 certificates support make XtreemFS usable over public networks. XtreemFS has been under development since early 2007. A first public release was made in August 2008. XtreemFS 1.0 was released in August 2009. The 1.0 release includes support for read-only replication with failover, data cent ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
