Inference flow orchestration service

    公开(公告)号:US11900169B1

    公开(公告)日:2024-02-13

    申请号:US17230784

    申请日:2021-04-14

    CPC classification number: G06F9/505 G06F16/9024 G06N20/00

    Abstract: Descriptors of machine learning tasks to be used to respond to analysis requests, indicating acceptable categories of runtime environments for the tasks and metrics to be collected from the tasks, are received via programmatic interfaces. In response to an analysis request, an orchestrator receives results from individual tasks as they become available, provides the results to other tasks, and causes a response to the request to be prepared using results from at least a subset of the tasks. Metrics collected from the tasks, and a visual representation of the tasks indicating their runtime environments are presented.

    Interface latency estimation based on platform subcomponent parameters

    公开(公告)号:US11366660B1

    公开(公告)日:2022-06-21

    申请号:US16447882

    申请日:2019-06-20

    Inventor: Anand Dhandhania

    Abstract: An API latency estimation system estimates latencies as a function of subcomponent parameters. The system may obtain first information indicative of at least a characteristic of data of a request provided to an API and second information indicative of at least a utilization of a first subcomponent of the API used to fulfill a subtask of a task of the request. An estimated latency for the first subcomponent to fulfill the subtask is determined at least in part by applying a latency estimation model for the API to at least the first information and the second information. If a comparison of the estimated latency to a measured latency for the first subcomponent to perform the subtask indicates a potential anomaly, then an indication of the potential anomaly may be outputted. The model may be updated with API request fulfillment data that is not anomalous.

    Inference flow orchestration service

    公开(公告)号:US12197958B2

    公开(公告)日:2025-01-14

    申请号:US18408405

    申请日:2024-01-09

    Abstract: Descriptors of machine learning tasks to be used to respond to analysis requests, indicating acceptable categories of runtime environments for the tasks and metrics to be collected from the tasks, are received via programmatic interfaces. In response to an analysis request, an orchestrator receives results from individual tasks as they become available, provides the results to other tasks, and causes a response to the request to be prepared using results from at least a subset of the tasks. Metrics collected from the tasks, and a visual representation of the tasks indicating their runtime environments are presented.

    INFERENCE FLOW ORCHESTRATION SERVICE
    5.
    发明公开

    公开(公告)号:US20240152402A1

    公开(公告)日:2024-05-09

    申请号:US18408405

    申请日:2024-01-09

    CPC classification number: G06F9/505 G06F16/9024 G06N20/00

    Abstract: Descriptors of machine learning tasks to be used to respond to analysis requests, indicating acceptable categories of runtime environments for the tasks and metrics to be collected from the tasks, are received via programmatic interfaces. In response to an analysis request, an orchestrator receives results from individual tasks as they become available, provides the results to other tasks, and causes a response to the request to be prepared using results from at least a subset of the tasks. Metrics collected from the tasks, and a visual representation of the tasks indicating their runtime environments are presented.

    Framework for building, orchestrating and deploying large-scale machine learning applications

    公开(公告)号:US11373119B1

    公开(公告)日:2022-06-28

    申请号:US16369884

    申请日:2019-03-29

    Abstract: Techniques for a framework for building, orchestrating, and deploying complex, large-scale Machine Learning (ML) or deep learning (DL) inference applications is described. A ML application orchestration service is disclosed that enables the construction, orchestration, and deployment of complex ML inference applications in a provider network. The disclosed service provides customers with the ability to define machine learning (ML) models and define transformation operations on data before and/or after being provided to the ML models to construct a complex ML inference application. The service provides a framework for the orchestration (co-ordination) of the workflow logic (e.g., of the request and/or response flows) involved in building and deploying a complex ML inference application in the provider network.

Patent Agency Ranking