摘要:
A scheduler with admissions control in a continuous media file server is presented. The scheduler supports multiple classes of tasks with diverse performance requirements. The scheduler is based on a combination of rate-monotonic and weighted round-robin scheduling schemes. Scheduling is accomplished in a hierarchical manner. Isochronous tasks have the highest priority and are scheduled first followed by real-time and general-purpose tasks. Isochronous tasks run periodically and are invoked by a timer interrupt set for each task. After scheduling the isochronous tasks, the scheduler alternates between the real-time tasks and the general-purpose tasks using a weighted round-robin scheme.
摘要:
A video file server includes an integrated cached disk array storage subsystem and a plurality of stream server computers linking the cached disk storage system to the data network for the transfer of video data streams. The video file server further includes a controller server for applying an admission control policy to client requests and assigning stream servers to service the client requests. The stream servers include a real-time scheduler for scheduling isochronous tasks, and supports at least one industry standard network file access protocol and one file access protocol for continuous media file access. The cached disk storage subsystem is responsive to video prefetch commands, and the data specified for a prefetch command for a process are retained in an allocated portion of the cache memory from the time that the cached disk storage subsystem has responded to the prefetch command to the time that the cached disk storage subsystem responds to a fetch command specifying the data for the process. The time between prefetching and fetching is selected based on available disk and cache resources. The video file server provides video-on-demand service by maintaining and dynamically allocating sliding windows of video data in the random access memories of the stream server computers.
摘要:
A file server includes active storage containing frequently accessed files, and active/inactive disk drives for containing infrequently accessed files. Groups of the files having become inactive in the active storage are successively migrated to respective evacuated active/inactive disk drives so that each active/inactive disk drive is loaded with files having a similar probability of access when access to the active/inactive disk drive reaches an inactivity threshold for powering down the active/inactive disk drive. Storage of the active/inactive disk drives is reclaimed when an oldest group of the files is archived or when an active/inactive disk drive is evacuated by migrating files from the active/inactive disk drive to storage having been released in other disk drives by promotion of files for client access to the promoted files. Therefore, recovery of storage can be planned and scheduled in advance and performed efficiently in a background process.
摘要:
A conventional network file server has a file system that permits file attributes and file data to be written in any order. The conventional network file server may also support an asynchronous write protocol, in which file attributes and file data need not be written to disk storage until a client sends a commit request. This asynchronous write protocol has a data security problem if the attributes are written before the data and the server crashes before completing the writing of the data to disk storage. This security problem is solved by adding a file system cache and following a protocol that writes the attributes to storage after writing the data to storage. For example, the attributes and data are stored in the file system cache and are not written down to storage until receipt of a commit request. When the commit request is received, the data is sent first from the file system cache to storage. Then the attributes are sent from the file system cache to storage. Then the file server acknowledges completion of the commit operation. In a preferred embodiment, storage is provided by an integrated cached disk array (ICDA) having a buffer cache and an array of disk drives, and the file system cache is distributed in a plurality of data mover computers interfaced to the ICDA. The addition of the file system cache to solve the security problem also reduces the burden on the buffer cache in the ICDA.
摘要:
Techniques are provided for storing files in a parallel computing system using different resolutions. A method is provided for storing at least one file generated by a distributed application in a parallel computing system. The file comprises one or more of a complete file and a sub-file. The method comprises the steps of obtaining semantic information related to the file; generating a plurality of replicas of the file with different resolutions based on the semantic information; and storing the file and the plurality of replicas of the file in one or more storage nodes of the parallel computing system. The different resolutions comprise, for example, a variable number of bits and/or a different sub-set of data elements from the file. A plurality of the sub-files can be merged to reproduce the file.
摘要:
A network file server includes a first set of data processors for receiving requests from clients, and a second set of data processors for accessing read-write file systems. A respective data processor in the second set is assigned to each file system for exclusive management of read and write locks on the file system. Each data processor in the first set can authorize access to file systems directly accessed by more than one data processor in the second set. Processing of a request for access that is authorized is continued by the data processor that is assigned to manage the locks on the file system to be accessed. The exclusivity of lock management eliminates cache coherency problems, and dynamic load balancing can be used to prevent the lock management from becoming a bottleneck to performance. A preferred embodiment includes a cached disk storage subsystem linked to data mover computers. Each data mover has authorization and ownership information for a plurality of file systems, and maintains a local cache of locking information for file systems that the data mover can directly access. The data mover receives a client request, and if access is authorized, checks whether it can directly access the file, and if not, forwards the request to another data mover.
摘要:
A plurality of data mover computers control access to respective file systems in data storage. A network client serviced by any of the data movers can access each of the file systems. If a data mover receives a client request for access to a file in a file system to which access is controlled by another data mover, then the data mover that received the client request sends a metadata request to the data mover that controls access to the file system. The data mover that controls access to the file system responds by placing a lock on the file and returning metadata of the file. The data mover that received the client request uses the metadata to formulate a data access command that is used to access the file data in the file system over a bypass data path that bypasses the data mover computer that controls access to the file system.
摘要:
A network file server includes a first set of data processors for receiving requests from clients, and a second set of data processors for accessing read-write file systems. A respective data processor in the second set is assigned to each file system for exclusive management of read and write locks on the file system. Each data processor in the first set can authorize access to file systems directly accessed by more than one data processor in the second set. Processing of a request for access that is authorized is continued by the data processor that is assigned to manage the locks on the file system to be accessed. The exclusivity of lock management eliminates cache coherency problems, and dynamic load balancing can be used to prevent the lock management from becoming a bottleneck to performance. A preferred embodiment includes a cached disk storage subsystem linked to data mover computers. Each data mover has authorization and ownership information for a plurality of file systems, and maintains a local cache of locking information for file systems that the data mover can directly access. The data mover receives a client request, and if access is authorized, checks whether it can directly access the file, and if not, forwards the request to another data mover.
摘要:
A network file server includes an integrated cached disk array and a plurality of data mover computers linking the cached disk array to the data network for file access. Each data mover computer maintains a local cache of file directory information including locking information of locked files that are accessible through the data mover. A cache consistency scheme ensures that shared locking information is consistent in the local caches. In a preferred embodiment, the cache consistency scheme uses an internal data link to pass messages between the data movers. The file directory information in the local cache includes information mapping file names to a list of logical blocks for each file name, file attributes that affect data access, and locking information down to a block level of granularity. Each data mover computer has network file manager software that maintains client ownership of locked files and communicates with the clients, and data mover file manager software that manages data mover computer ownership of locked files and communicates with other data mover computers. The cache consistency messages indicate a grant of data mover ownership over a file, a release of data mover ownership over a file, a denial of a request for access to a locked file, and changes to file attributes affecting access to a file. The local caches and the cache consistency scheme eliminate any need to store file locking information in the cached disk array.
摘要:
A video file server includes an integrated cached disk array storage subsystem and a plurality of stream server computers linking the cached disk array to a data network for the transfer of video data streams. The video file server further includes a controller server for applying an admission control policy to client requests and assigning stream servers to service the client requests. The stream servers include a real-time scheduler for scheduling isochronous tasks, and supports at least one industry standard network file access protocol and one file access protocol for continuous media file access. The cached disk storage subsystem is responsive to video prefetch commands, and the data specified for a prefetch command for a process are retained in an allocated portion of the cache memory from the time that the cached disk storage subsystem has responded to the prefetch command to the time that the cached disk storage subsystem responds to a fetch command specifying the data for the process. The time between prefetching and fetching is selected based on available disk and cache resources. The video file server provides video-on-demand service by maintaining and dynamically allocating sliding windows of video data in the random access memories of the stream server computers. The video file server has a tape silo for providing network backup services, and data to be written to tape are prestaged in the cached disk storage subsystem.