Overview

WorksAudit is intended to be a SaaS, so the system would have to be:

1. Infinitely scalable.
2. Cost as little as possible when unused.
3. Can support any kind of system’s audit requirements:
   1. Should be designed in such a way that it is able to receive, process, and store any kind of audit log data.
   2. Should have the APIs for both end-user applications and in-house Audit Log Viewer to consume all the data being fed into the system.
4. Multi-tenant, multi-landscape.

WorksAudit composed of hundreds of components (implemented as AWS resources), and it would be overwhelming to show them all in one picture. However, to understand the basic idea on how WorksAudit is structured is actually quite simple. By understanding this underlying idea, understanding the detail parts would become easier.

Following image is the simplified architecture of WorksAudit that shows the main idea of how the whole system is constructed:

This diagram can be explained as follows:

1. There are 3 main areas that are the focus of an audit system:
   1. Systems to be audited, shown above on the left side. These systems are producing audit logs and send them to the second part of the system.
   2. The area in the middle that receives the log data from the systens describe above. The purpose of this part is:
      1. To transform the log data coming in to make sure that the data can be usable further.
      2. To store the data in a suitable format and storage.
      3. To make the data easily available to use by auditor.
   3. Systems used by auditor (end-user) shown on the right side. These systems consumes data processed, stored, and presented by the middle area described above.
2. The left area are not systems that are developed and maintained by WorksAudit team. However, those systems used SDK developed by WorksAudit team. These SDKs are usually called Producer.

The simplified architecture described in the previous section is shown in the following diagram as how it is implemented as AWS resources. This diagram is still simplified and not showing the actual detail implementations, but this may help to understand the approach of how WorksAudit is structured in AWS:

The diagram can be read from left to right. The whole flow can be summarized as follows:

1. The systems to be audited send the log data using Producer SDK. This Producer SDK is producing logs in Protobuf format.
2. The logs are sent through AWS Kinesis Firehose to be stored in WorksAudit central bucket (wap-audit-central-{env} where {env} is the environment name).
3. The data collected in WorksAudit central bucket is then processed by data transformation process (ETL) that is executed as a job in AWS Glue.
4. The ETL job produces a database in Parquet format to be used by Athena. This database is stored in WorksAudit database bucket (wap-audit-db-{env}).
5. Other than log data, there are some other data synchronized from other systems:
   1. User authorization data.
   2. Multi-lingual data.
6. The data in the databases then will be fetched by various APIs that exposes the data to the external consumer:
   1. Query (GraphQL) API is the main API for log data search and retrieval used by the official WorksAudit Log Viewer.
   2. Query Lite (REST) API is the API mainly directed to be used for direct CSV data search download by end-user scripts or backend systems.
   3. Reference API is the API for getting mainly static reference data, for example multilingual names. However this API also exposes some dynamic data such as sytem status.
   4. Auth API is the API for authentication and authorization-related tasks. All API endpoints require authentication, and this set of API is the one to use for establishing an authenticated connection.
   5. Settings API is the API for retrieving and updating systems' settings.
7. To simplify the access to the APIs on client applications, we also develop an SDK called Consumer SDK.
8. The Audit Log Viewer is a single-page application (SPA) where the end-user (customer) can search and view the result of the whole process.

1. Activity Specification. This contains the master list of all activity types supported by WorksAudit and the external systems that produced the activities.
2. HUE Producer Specification. This specifies the details of the data to be sent by HUE producers.
3. Collabo/WorksSuite Producer Specification. This specifies the details of the data to be sent by Collabo/WorksSuite producers.
4. EBM Producer Specification. This specifies the details of the data to be sent by EBM producers.
5. AC Producer Specification. This specifies how to map AC activities to WorksAudit’s.
6. Expense Producer Specification. This specifies how to map Expense activities to WorksAudit’s.

1. Query API Specification.
2. Query Lite API Specification.
3. Reference API Specification.

1. ID Structure Specification. This document specifies how the IDs in WorksAudit is structured, and how it relates to the IDs in the log data source systems.