摘要:
A storage scheme allocates portions of a logical volume to storage nodes in excess of the capacity of the storage nodes. Slices of the storage nodes and segments of slices are allocated in response to write requests such that actual allocation on the storage nodes is only in response to usage. Segments are identified with virtual segment identifiers that are retained when segments are moved to a different storage node. Logical volumes may therefore be moved seamlessly to different storage nodes to ensure sufficient storage capacity. Data is written to new locations in segments having space and a block map tracks the last segment to which data for a given address is written. Garbage collection is performed to free segments that contain invalid data, i.e. data for addresses that have been subsequently written to.
摘要:
A new snapshot of a storage volume is created by suppressing write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot. Subsequent write requests to the storage volume are then performed on the segments allocated to the new snapshot. An orchestration layer implements a bundled application that is provisioned with storage volumes and containers. A snapshot of the application may be created and used to rollback or clone the application. The amount of processing cores, memory, and containers may be increased or decreased based on usage. Components of the bundled application may be assigned to nodes to satisfy affinity and anti-affinity rules. Device sets may be generated according to these rules and used to plan for the assignment of storage volumes and containers of the bundled application.
摘要:
A system provides systems and methods for visualizing a computational process. A portion of attributes of a computational process may be displayed on a face of a three-dimensional object. In response to rotation of the object by a user, a different face of the object may be displayed and a different portion of the attributes displayed thereon. Each object may represent a status of a computational cluster, specifically a set of jobs executing on the cluster. A cluster may be selected and the jobs represented by means of graphical indicators arranges on a time axis and a performance axis. The size of a graphical indicator may indicate an amount of processing time consume by the corresponding job. A color of the graphical indicator may indicate a status. A portion of the graphical indicator may be visually distinguished from a remaining portion to indicate a completion percentage of the job.
摘要:
A system maintains a consistency database that maintains a status (current, down, stale) for copies of logical storage volumes stored on storage nodes. As failures are detected, the consistency database is updated. Copies are synchronized with one another using information in the consistency database. Write operations on a primary node for a slice of a logical storage node are assigned a virtual block address (VBA) that is mapped to a logical block address (LBA) within the slice. Consistency of the VBAs of the primary node and that of a secondary node is evaluated and used to detect currency. VBA holes are detected and corresponding write commands resent to maintain currency. Physical segments on the primary node are assigned virtual segment identifiers (VSID) that are maintained consistent with VSIDs on clone nodes so that they can be used for garbage collection and synchronization.
摘要:
A new snapshot of a storage volume is created by instructing computing nodes to suppress write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot and finalizes and performs garbage collection with respect to segments allocated to the previous snapshot. Subsequent write requests to the storage volume are then performed on the segments allocated to the new snapshot. An orchestration layer implements a multi-role application that is provisioned with virtualized storage and computation resources. A snapshot of the application may be created and used to rollback or clone the application. Clones snapshots of storage volumes may be gradually populated with data from prior snapshots to reduce loading on a primary snapshot. Upgrades for bundled applications may be defined and executed for containers and roles of the bundled application.
摘要:
A new snapshot of a storage volume is created by instructing computing nodes to suppress write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot and finalizes and performs garbage collection with respect to segments allocated to the previous snapshot. Subsequent write requests to the storage volume are then performed on the segments allocated to the new snapshot. An orchestration layer implements a multi-role application that is provisioned with virtualized storage and computation resources. A snapshot of the application may be created and used to rollback or clone the application. Containers may be loaded with data structures for encrypted and/or authenticated communication. Failed storage volumes maybe detected and repaired. Repopulating repaired volumes may be managed by an application or a storage manager implementing a virtualized storage system.
摘要:
A new snapshot of a storage volume is created by instructing computing nodes to suppress write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot and finalizes and performs garbage collection with respect to segments allocated to the previous snapshot. Subsequent write requests to the storage volume are then performed on the segments allocated to the new snapshot. An orchestration layer implements a multi-role application that is provisioned with virtualized storage and computation resources. A snapshot of the application may be created and used to rollback or clone the application. Clones snapshots of storage volumes may be gradually populated with data from prior snapshots to reduce loading on a primary snapshot.
摘要:
A system maintains a consistency database that maintains a status (current, down, stale) for copies of logical storage volumes stored on storage nodes. As failures are detected, the consistency database is updated. Copies are synchronized with one another using information in the consistency database. Write operations on a primary node for a slice of a logical storage node are assigned a virtual block address (VBA) that is mapped to a logical block address (LBA) within the slice. Consistency of the VBAs of the primary node and that of a secondary node is evaluated and used to detect currency. VBA holes are detected and corresponding write commands resent to maintain currency. Physical segments on the primary node are assigned virtual segment identifiers (VSID) that are maintained consistent with VSIDs on clone nodes so that they can be used for garbage collection and synchronization.
摘要:
Creation of a virtual overlay for a multipath network with tiered topology is disclosed. A virtual-node module may group network nodes connected within a common tier, which is initially the tier furthest from the core. This module may also group network nodes at a consecutive tier, closer to the core, that connect a pairs of groupings in the common tier by links between the common and consecutive tiers. Groupings with a common network node may be combined and designated as virtual nodes. A virtual-link module may sum bandwidths for network links between virtual-node pairs with nodes in both the common and consecutive tiers. A check module may combine virtual nodes and links and network nodes and links not yet virtualized in a graph, which it may check for a cycle. An iteration module may iterate both tiers toward the core, triggering further grouping, summing, and updating until no loop exists.
摘要:
Systems and methods are disclosed for scheduling jobs processed in a distributed fashion to realize unharnessed efficiencies latent in the characteristics of the jobs and distributed processing technologies. A job store may be communicatively coupled to a job analyzer. The job analyzer may be operable to read information characterizing a job to identify multiple data blocks to be processed during the job at multiple locations in a cluster of nodes. A scheduling module may use information about the multiple data blocks, their storage locations, their status with respect to being provisioned to processing logic, data blocks to be processed by other jobs, data blocks in cache that have been pre-fetched for a prior job, quality-of-services parameters, and/or job characteristics, such as job size, to schedule the job in relation to other jobs.