Abstract:
Commitments against various resources can be dynamically adjusted for customers in a shared-resource environment. A customer can provision a data volume with a committed rate of Input/Output Operations Per Second (IOPS) and pay only for that commitment (plus any overage), for example, as well as the amount of storage requested. The customer can subsequently adjust the committed rate of IOPS by submitting an appropriate request, or the rate can be adjusted automatically based on any of a number of criteria. Data volumes for the customer can be migrated, split, or combined in order to provide the adjusted rate. The interaction of the customer with the data volume does not need to change, independent of adjustments in rate or changes in the data volume, other than the rate at which requests are processed.
Abstract:
A set of virtualized computing services may include multiple types of virtualized data store differentiated by characteristics such as latency, throughput, durability and cost. A sequence of captures of a data set from one data store to another may be scheduled to achieve a variety of virtualized computing service user and provider goals such as lowering a probability of data loss, lowering costs, and computing resource load leveling. Data set captures may be scheduled according to policies specifying fixed and flexible schedules and conditions including flexible scheduling windows, target capture frequencies, probability of loss targets and/or cost targets. Capture lifetimes may also be managed with capture retention policies, which may specify fixed and flexible lifetimes and conditions including cost targets. Such data set capture policies may be specified with a Web-based administrative interface to a control plane of the virtualized computing services.
Abstract:
Virtual data stores may be sparsely provisioned by virtual data storage services in a manner that controls risk of implementation resource shortages. Relationships between requested data storage space size, data storage server capacity, allocated data storage space size and/or allocated data storage space utilization may be tracked on a per data store, per customer, per data storage server, and/or a per virtual data storage service basis. For each such basis, a set of constraints may be specified to control the relationships. The set of constraints may be enforced during implementation resource allocation, and by migration of data storage space portions to different implementation resources as part of a sparse provisioning load balancing. Sparse provisioning details may be made explicit to virtual data storage service customers to varying degrees including explicit, aggregate on a per customer basis, and aggregate on a per virtual data storage service basis.
Abstract:
With the advent of virtualization technologies, networks and routing for those networks can now be simulated using commodity hardware rather than actual routers. For example, virtualization technologies such as those provided by VMWare, XEN, or User-Mode Linux can be adapted to allow a single physical computing machine to be shared among multiple virtual networks by providing each virtual network user with one or more virtual machines hosted by the single physical computing machine, with each such virtual machine being a software simulation acting as a distinct logical computing system that provides users with the illusion that they are the sole operators and administrators of a given hardware computing resource. In addition, routing can be accomplished through software, providing additional routing flexibility to the virtual network in comparison with traditional routing. As a result, in some implementations, supplemental information other than packet information can be used to determine network routing.
Abstract:
A set of virtualized computing services may include multiple types of virtualized data store differentiated by characteristics such as latency, throughput, durability and cost. A sequence of captures of a data set from one data store to another may be scheduled to achieve a variety of virtualized computing service user and provider goals such as lowering a probability of data loss, lowering costs, and computing resource load leveling. Data set captures may be scheduled according to policies specifying fixed and flexible schedules and conditions including flexible scheduling windows, target capture frequencies, probability of loss targets and/or cost targets. Capture lifetimes may also be managed with capture retention policies, which may specify fixed and flexible lifetimes and conditions including cost targets. Such data set capture policies may be specified with a Web-based administrative interface to a control plane of the virtualized computing services.
Abstract:
Techniques are described for managing access of executing programs to non-local block data storage. In some situations, a block data storage service uses multiple server storage systems to reliably store network-accessible block data storage volumes that may be used by programs executing on other physical computing systems. A group of multiple server block data storage systems that store block data volumes may in some situations be co-located at a data center, and programs that use volumes stored there may execute on other physical computing systems at that data center. If a program using a volume becomes unavailable, another program (e.g., another copy of the same program) may in some situations obtain access to and continue to use the same volume, such as in an automatic manner in some such situations.
Abstract:
Patterns of access and/or behavior can be analyzed and persisted for use in pre-fetching data from a physical storage device. In at least some embodiments, data can be aggregated across volumes, instances, users, applications, or other such entities, and that data can be analyzed to attempt to determine patterns for any of those entities. The patterns and/or analysis can be persisted such that the information is not lost in the event of a reboot or other such occurrence. Further, aspects such as load and availability across the network can be analyzed to determine where to send and/or store data that is pre-fetched from disk or other such storage in order to reduce latency while preventing bottlenecks or other such issues with resource availability.
Abstract:
Various aspects of a data volume or other shared resource are determined and updated dynamically for purposes such as to provide guaranteed qualities of services. For example, the number of partitions in a data volume and/or the way in which data is stored across those partitions can be updated dynamically without significantly impacting the customer using the volume. The data stored to the volume can be striped or otherwise distributed across a number of logical areas, which then can be distributed across the partitions. Separate mappings can be used for the data in each logical area, and the logical areas in each partition, such that when moving a logical area only a single mapping has to be updated, regardless of the amount of data in that logical area. Further, logical areas can be moved between partitions without the need to repartition or redistribute the data in the data volume.
Abstract:
A materialization configuration request is received via a programmatic interface from a client of a journal-based multi-data-store database. The request indicates a partitioning rule to be used to select, for respective writes indicated in committed transaction entries of a journal, the materialization node at which the writes are to be stored. A control plane component of the database verifies that a set of materialization nodes corresponding to the partitioning rule has been established, and initiates the propagation of writes from the journal to the materialization nodes by respective write appliers.
Abstract:
A new type system may be added to a type registry for a data processing service. A request to add the new type system may be received that describes the new type system for a data store. The new type system may be used to perform a data processing job that accesses the data store to obtain or store data as a source or target data store.