Abstract:
A self-adaptive control system based on proportional-integral (PI) control theory for dynamic capacity management of latency-sensitive application servers (e.g., application servers associated with a social networking application) are disclosed. A centralized controller of the system can adapt to changes in request rates, changes in application and/or system behaviors, underlying hardware upgrades, etc., by scaling the capacity of a cluster up or down so that just the right amount of capacity is maintained at any time. The centralized controller uses information relating to a current state of the cluster and historical information relating to past state of the cluster to predict a future state of the cluster and use that prediction to determine whether to scale up or scale down the current capacity to reduce latency and maximize energy savings. A load balancing system can then distribute traffic among the servers in the cluster using any load balancing methods.
Abstract:
Disclosed are a method and system for recovering a distributed system from a failure of a data storage unit. The distributed system includes a plurality of computer systems, each having a read-write computer and a data storage unit. Data is replicated from a particular data storage unit to other data storage units using publish-subscribe model. A read-write computer receives the replicated data, processes the data for any conflicts and stores it in the data storage unit. If a data storage unit fails, another data storage unit that has latest data corresponding to the failed data storage unit is determined and the latest data is replicated to other data storage units. Accordingly, the distributed system continues to have the data of the failed data storage unit. The failed data storage unit may be reconstructed using data from one of the other data storage units in the distributed system.
Abstract:
A self-adaptive control system based on proportional-integral (PI) control theory for dynamic capacity management of latency-sensitive application servers (e.g., application servers associated with a social networking application) are disclosed. A centralized controller of the system can adapt to changes in request rates, changes in application and/or system behaviors, underlying hardware upgrades, etc., by scaling the capacity of a cluster up or down so that just the right amount of capacity is maintained at any time. The centralized controller uses information relating to a current state of the cluster and historical information relating to past state of the cluster to predict a future state of the cluster and use that prediction to determine whether to scale up or scale down the current capacity to reduce latency and maximize energy savings. A load balancing system can then distribute traffic among the servers in the cluster using any load balancing methods.
Abstract:
When multiple regional data clusters are used to store data in a system, maintaining cache consistency across different regions is important for providing a desirable user experience. In one embodiment, there is a master data cluster where all data writes are performed, and the writes are replicated to each of the slave data clusters in the other regions. Appended to the replication statements are invalidations for cache values for the keys whose values have been changed in the master data cluster. An apparatus in the master data cluster logs replication statements sent to the slave databases. When a slave database fails, the apparatus extracts the invalidations intended for the failed database and publishes the invalidations to a subscriber in the region of the failed database. The subscriber sends the invalidations to the local caches to cause stale data for those keys to be deleted from the caches.
Abstract:
A self-adaptive control system based on proportional-integral (PI) control theory for dynamic capacity management of latency-sensitive application servers (e.g., application servers associated with a social networking application) are disclosed. A centralized controller of the system can adapt to changes in request rates, changes in application and/or system behaviors, underlying hardware upgrades, etc., by scaling the capacity of a cluster up or down so that just the right amount of capacity is maintained at any time. The centralized controller uses information relating to a current state of the cluster and historical information relating to past state of the cluster to predict a future state of the cluster and use that prediction to determine whether to scale up or scale down the current capacity to reduce latency and maximize energy savings. A load balancing system can then distribute traffic among the servers in the cluster using any load balancing methods.
Abstract:
Disclosed are a method and system for recovering a distributed system from a failure of a data storage unit. The distributed system includes a plurality of computer systems, each having a read-write computer and a data storage unit. Data is replicated from a particular data storage unit to other data storage units using publish-subscribe model. A read-write computer receives the replicated data, processes the data for any conflicts and stores it in the data storage unit. If a data storage unit fails, another data storage unit that has latest data corresponding to the failed data storage unit is determined and the latest data is replicated to other data storage units. Accordingly, the distributed system continues to have the data of the failed data storage unit. The failed data storage unit may be reconstructed using data from one of the other data storage units in the distributed system.
Abstract:
Disclosed is a technology for publishing subscriptions in a publish-subscribe system in accordance with subscription groups. The technology may include (i) determining a consumption characteristic by which each of multiple subscribers in a publish-subscribe system consumes a subscription published by a publisher; (ii) identifying the subscribers whose consumption characteristics satisfy a specified similarity criterion; (iii) defining multiple subscription groups, each of which includes subscriptions of those of the subscribers whose consumption characteristics satisfy the specified similarity criterion; (iv) storing the subscriptions in multiple logical partitions of a storage system where each of the logical partitions contains a separate non-overlapping subset of the subscriptions; and (v) transmitting the subscriptions to the subscribers in accordance with the subscription groups.
Abstract:
Disclosed is a technology for publishing subscriptions in a publish-subscribe system in accordance with subscription groups. The technology may include (i) determining a consumption characteristic by which each of multiple subscribers in a publish-subscribe system consumes a subscription published by a publisher; (ii) identifying the subscribers whose consumption characteristics satisfy a specified similarity criterion; (iii) defining multiple subscription groups, each of which includes subscriptions of those of the subscribers whose consumption characteristics satisfy the specified similarity criterion; (iv) storing the subscriptions in multiple logical partitions of a storage system where each of the logical partitions contains a separate non-overlapping subset of the subscriptions; and (v) transmitting the subscriptions to the subscribers in accordance with the subscription groups.
Abstract:
Disclosed is a technology for publishing subscriptions in a publish-subscribe system in accordance with subscription groups. The technology may include (i) determining a consumption characteristic by which each of multiple subscribers in a publish-subscribe system consumes a subscription published by a publisher; (ii) identifying the subscribers whose consumption characteristics satisfy a specified similarity criterion; (iii) defining multiple subscription groups, each of which includes subscriptions of those of the subscribers whose consumption characteristics satisfy the specified similarity criterion; (iv) storing the subscriptions in multiple logical partitions of a storage system where each of the logical partitions contains a separate non-overlapping subset of the subscriptions; and (v) transmitting the subscriptions to the subscribers in accordance with the subscription groups.
Abstract:
When multiple regional data clusters are used to store data in a system, maintaining cache consistency across different regions is important for providing a desirable user experience. In one embodiment, there is a master data cluster where all data writes are performed, and the writes are replicated to each of the slave data clusters in the other regions. Appended to the replication statements are invalidations for cache values for the keys whose values have been changed in the master data cluster. An apparatus in the master data cluster logs replication statements sent to the slave databases. When a slave database fails, the apparatus extracts the invalidations intended for the failed database and publishes the invalidations to a subscriber in the region of the failed database. The subscriber sends the invalidations to the local caches to cause stale data for those keys to be deleted from the caches.