SAVING PRODUCTION RUNS OF A FUNCTION AS UNIT TEST AND AUTOMATIC OUTPUT REGENERATION

    公开(公告)号:US20240403634A1

    公开(公告)日:2024-12-05

    申请号:US18676339

    申请日:2024-05-28

    Abstract: An artificial intelligence system can be used to respond to natural language inputs. The AI System may, for example, receive a first user input for a LLM, generate a first prompt based on the first user input, transmit the first prompt to an LLM, receive an output from the LLM, and evaluate the output from the LLM with reference to one or more validation tests. Responsive to determining that the output from the LLM is not validated, generate a second prompt for the LLM, where the second prompt indicates at least an aspect of the output that caused the output to not be evaluated (e.g., a portion of the output that may need to be updated or corrected), transmit the second prompt to the LLM, and receive an updated output from the LLM. The AI system can include an application for testing functions that utilize interactions with language models.

    LARGE DATA SET MANAGEMENT WITH LARGE LANGUAGE MODELS

    公开(公告)号:US20240403289A1

    公开(公告)日:2024-12-05

    申请号:US18674628

    申请日:2024-05-24

    Abstract: A system may receive a natural language query. A system may receive indications of one or more data object types, wherein each of the one or more data object types is associated with a respective one or more properties. A system may receive references to one or more data sets, wherein the one or more data sets are each associated with at least a respective data object type. A system may transmit a prompt to a large language model (“LLM”), the prompt comprising at least: the natural language query, the indications of the one or more data object types, and the references to the one or more data sets. A system may receive, from the LLM, a response to the prompt, wherein the response includes indications of: at least a first reference to a first data set and a query to be applied to the first data set.

    STRUCTURING AND RICH DEBUGGING OF INPUTS AND OUTPUTS TO LARGE LANGUAGE MODELS

    公开(公告)号:US20240403194A1

    公开(公告)日:2024-12-05

    申请号:US18676141

    申请日:2024-05-28

    Abstract: The disclosure is directed to methods and systems for improving interactions with a Large Language Model (LLM). An artificial intelligence system (AIS) can receive user inputs via a graphical user interface indicating a task to be performed by the LLM, one or more tools which may be accessed by the AIS in response to tool calls from the LLM, and an output schema for structuring a format of a response from the LLM. The AIS can generate a prompt for the LLM based on the user input. The prompt can include indications of the one or more tools, one or more example tool operations, the task to be performed, and an indication of the output schema. The AIS can include a debugging application or module enabling rich debugging of language model interactions in a single view.

    SECURING LARGE LANGUAGE MODEL OUTPUT BY PROPAGATING PERMISSIONS

    公开(公告)号:US20240403396A1

    公开(公告)日:2024-12-05

    申请号:US18675587

    申请日:2024-05-28

    Abstract: Computer-implemented systems and methods are disclosed, including for determining permissions for nondeterministic model output. A computer-implemented method may include, for example, receiving one or more user inputs including a first user input providing at least a portion of a first prompt for a query for a first nondeterministic model. A computer-implemented method may in response to receiving the one or more user inputs include: executing the query, by the first nondeterministic model, to generate an output, determining a first one or more data inputs used by the first nondeterministic model during execution of the query, determining a first set of permissions associated with the first one or more data inputs; and applying a second set of permissions to at least a first portion of the output based on the first set of permissions.

    SECURE HIGH SCALE CRYPTOGRAPHIC COMPUTATION THROUGH DELEGATED KEY ACCESS

    公开(公告)号:US20230418953A1

    公开(公告)日:2023-12-28

    申请号:US17846648

    申请日:2022-06-22

    CPC classification number: G06F21/602 H04L9/088

    Abstract: An apparatus, computer-implemented method and computer program are disclosed for performing a cryptographic operation in a high-trust (HT) environment. The HT environment including a compute service and key storage service. The compute service receives from a user device, a user request for performing a cryptographic operation on at least a portion of a large-scale dataset. The user request including a user token associated with a user of the user device. The compute service sends to the key storage service, a cryptographic key access request corresponding to the received user request. The cryptographic key access request including data representative of the user token and/or a compute service token. The key storage service determines from the user token and/or compute service token whether the user has permission to have the cryptographic operation performed and/or whether to grant the compute service access to data representative of the cryptographic key in relation to the requested cryptographic operation when user has permission. In response to the key storage service granting access to the compute service, the key storage service sends to the compute service the requested cryptographic key/algorithm associated with the cryptographic operation of the user request. The compute service performs the cryptographic operation on the portion of the large-scale dataset based on the received cryptographic key/algorithm.

Patent Agency Ranking