Intelligent Web Automation

Maintaining legal compliance is crucial as it ensures adherence to laws and regulations, mitigates risks, and protects organizations from legal consequences such as fines and lawsuits. Compliance also plays a key role in safeguarding an organization's reputation, fostering trust among stakeholders, and supporting business continuity by preventing disruptions arising from regulatory non-compliance. This example shows how to implement process to comply with EU regulations by reviewing all new legislations published at https://eur-lex.europa.eu/oj/direct-access.html

• How does it work?

  1. Open EU Regulation page with new legislation acts
  2. Open the most recent entry
  3. Open each act one by one
  4. Check if the new legislation act URL is already in the database
  5. Check if the act is available in English
  - If it is then create a short summary using OpenAI
  - If not then save the act URL and summary as "English version is not available"
  

• Process Drawing

  flowchart TD; Start -->|Click on page|Open_Page; Open_Page["Open Page Actions: Browser opens the web page Participants: Browser, User IT Systems: Browser"]; Open_Page -->|"Page is open"|Open_Latest_Legislation; Open_Page ~~~|"User opens the page. Browser loads the page and displays the content to the user"|Open_Page; Open_Latest_Legislation["Open Latest Legislation Act Actions: User opens the latest legislation acts Participants: User, Browser IT Systems: Browser, Website"]; Open_Latest_Legislation -->|"URL of the first legislation act is obtained"|Check_URL_Database; Open_Latest_Legislation ~~~|"User navigates to the part of the page with the latest legislation acts. URL of the first act is obtained"|Open_Latest_Legislation; Check_URL_Database["Check URL in Database Actions: System checks if URL exists in database Participants: Database IT Systems: Database"]; Check_URL_Database -->|"URL exists in the database"|Rerun_Process_for_Next_Act; Check_URL_Database -->|"URL doesn't exist in the database"|Open_Act_Page; Check_URL_Database ~~~|"System checks the database if the URL of the first legislation act already exists"|Check_URL_Database; Open_Act_Page["Open Act Page Actions: Browser opens the act page from URL Participants: User, Browser IT Systems: Browser, Website"]; Open_Act_Page -->|"Act page is opened"|Check_English_Availability; Open_Act_Page ~~~|"Browser loads the page of the act and displays the content to the user"|Open_Act_Page; Check_English_Availability["Check if Act is Available in English Actions: System checks the language of the act Participants: System IT Systems: Browser, Website"]; Check_English_Availability -->|"Act not available in English"|Save_as_Not_Available; Check_English_Availability -->|"Act available in English"|Open_HTML_Version; Check_English_Availability ~~~|"System checks if the legislation act is available in English version"|Check_English_Availability; Save_as_Not_Available["Save URL as Not Available Actions: System saves URL and summary as not available Participants: Database IT Systems: Database"]; Save_as_Not_Available -->|"URL and summary saved"|Rerun_Process_for_Next_Act; Save_as_Not_Available ~~~|"System saves the URL and summary as 'English version is not available' in the database"|Save_as_Not_Available; Open_HTML_Version["Open HTML Version of Act Actions: Browser opens the HTML version of the act Participants: Browser, User IT Systems: Browser, Website"]; Open_HTML_Version -->|"HTML version of the act is opened"|Save_HTML_Text; Open_HTML_Version ~~~|"Browser loads the HTML version of the act and displays the content"|Open_HTML_Version; Save_HTML_Text["Save HTML Text Actions: System saves the HTML text Participants: Database IT Systems: Database"]; Save_HTML_Text -->|"HTML text saved"|OpenAI_Summary; Save_HTML_Text ~~~|"System saves the HTML text of the act in the database"|Save_HTML_Text; OpenAI_Summary["Generate Summary using OpenAI Actions: OpenAI plugin generates a short summary Participants: OpenAI Plugin IT Systems: OpenAI Plugin"]; OpenAI_Summary -->|"Summary generated"|Save_Summary; OpenAI_Summary ~~~|"OpenAI plugin processes the HTML text and generates a short summary of the act"|OpenAI_Summary; Save_Summary["Save URL and Summary Actions: System saves URL and summary to the database Participants: Database IT Systems: Database"]; Save_Summary -->|"URL and summary saved"|Rerun_Process_for_Next_Act; Save_Summary ~~~|"System saves the URL and summary of the act in the database"|Save_Summary; Rerun_Process_for_Next_Act["Rerun Process for Next Act Actions: System reruns the process for next act Participants: System IT Systems: Browser, Database"]; Rerun_Process_for_Next_Act -->|"Next act processed"|Open_Latest_Legislation; Rerun_Process_for_Next_Act -->|"No next act"|End; Rerun_Process_for_Next_Act ~~~|"System reruns the process for the next legislation act. If there is no next act, the process stops"|Rerun_Process_for_Next_Act; End; style Open_Page fill:#90ee90,stroke-width:2px; style Open_Latest_Legislation fill:#90ee90,stroke-width:2px; style Check_URL_Database fill:#90ee90,stroke-width:2px; style Open_Act_Page fill:#90ee90,stroke-width:2px; style Check_English_Availability fill:#90ee90,stroke-width:2px; style Save_as_Not_Available fill:#90ee90,stroke-width:2px; style Open_HTML_Version fill:#90ee90,stroke-width:2px; style Save_HTML_Text fill:#90ee90,stroke-width:2px; style OpenAI_Summary fill:#90ee90,stroke-width:2px; style Save_Summary fill:#90ee90,stroke-width:2px; style Rerun_Process_for_Next_Act fill:#90ee90,stroke-width:2px;

• Process requirements

  Process uses 'legislation_acts' MySQL database table

  CREATE TABLE `legislation_acts` (
    `id` bigint NOT NULL,
    `url` varchar(300) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
    `summary` varchar(1000) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
    `c` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP
  ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
  
  ALTER TABLE `legislation_acts`
    ADD PRIMARY KEY (`id`);
  
  ALTER TABLE `legislation_acts`
    MODIFY `id` bigint NOT NULL AUTO_INCREMENT;
  COMMIT;

• Process Design

  Process design is available on GitHub here.