Response Automation

Prerequisites:

RegML and Response Automation are both enabled by an administrator.
The RegML backend and vector database have been deployed to your environment.
Your RegScale environment is configured correctly to communicate with the other parts of the RegML Infrastructure.

Click Response Automation under Modules
Click the blue Start Response button in the upper right corner.
Enter a title to identify your Response Automation entry by.
Select the Word/LibreOffice, PDF, Excel, or CSV document that you wish to extract questions from to be uploaded.
Click Next
Search for the materials that you would like the language model to search for relevant information, when answering questions. Note: Currently only control implementation statements are available for searching, but additional data types such as files, policies, and previous response automation results are soon to follow.
Click the Respond button. You should receive a notification that your job has been submitted.
Refresh the page to be notified when the Response Automation job is finished processing.

Once a Response Automation job is Finished, you can drill down into the record and click “Responses” in the upper right menu.
Note that not every question is guaranteed to have found sufficient citations to answer the question.
From the list view of responses and questions, you can select and delete any questions that do not agree belong in the set.
Clicking the “More Options” (3 dots) menu on the right will allow you to open the Review Responses page.

Responses are reviewed in sets of 10. You can paginate through the responses by clicking the Previous or Next buttons at the bottom of the set of records.
The Citations window on the right shows the excerpts of text from your specified source documents that were used by the language model to answer this question. Take note that, due to processing requirements, these citations are often not neatly divided into complete sentences or paragraphs. A citation may begin or end with an incomplete sentence.
The WYSIWYG editor allows you to modify any response. Changes to the text must be saved using the Save button on the bottom right of the records review page.
The Accepted checkbox allows you to keep track of which responses you have accepted and approved. Acceptance status is purely for your information, and has no impact on program performance. Recall that overall Acceptance status is reported in aggregate at the List View of the Response Automation details page.
The Accept all button marks all the records on the current page as accepted. It does not affect records not currently shown on the page.
The Search box on the top right will filter displayed records based on the contents of the question or response.

When you are satisfied with the work, you may export the resulting questions and answers into an Excel file.

There is an error submitting the job-- This notification means the RegML backend cannot be reached to submit the job. Confirm the necessary infrastructure components are in place and that environment variables are set.
Response Automation Job Stuck at “In Progress” status – This means the RegML backend is unable to communicate back to RegScale to update the status of the job. Ensure that RegML infrastructure is both present in the environment and that the environment is correctly configured.
Response Automation incorrectly identified something as a question / Failed to identify a question – Language models sometimes make mistakes in recognizing when text is intended to represent a request for information— especially in cases where the language of the request is ambiguous or not clearly a question. New questions can manually be added to the Response Automation set. Please share any concerns or experiences with your customer service representative, so that we can use this information to improve model performance.