Documentation
- Introduction.
- Requirements.
- Installation.
- Configuration.
- Sending Audit Events.
- Implementing Meta Data class.
- Using Layouts.
- Data de-identification.
- Plugins.
- Custom Event Filters.
- Monitoring And Management.
- Unit testing with Audit4j.
- Performance Tuning.
- Plugin developments.
1. Introduction
Audit4j is an open source auditing framework which is a full stack application auditing solution for java enterprise projects. Refer the FAQ (frequently asked question) section for an overview of the application and the audit4j framework.
Audit4j is Open Source and available under the Apache 2 License.
Download from http://audit4j.org/downloads.
Code examples are prepared, which are included in a set of demo projects in order to demonstrate the usage of Audit4j and its plugins. For further information, please refer the following URL in order to access the project on Github.
http://github.com/audit4j/audit4j-demo
This documentation undergoes constant modifications (i.e.: added features, improvements, etc).
This document serves as the guide to the Audit4j framework. It explains the integration of the application, its configuration, optimization, security and usage of the plugin.
2. requirements
Audit4j has been tested on a common distributions of Linux, Windows and Mac OS. Even though Audit4j is entirely built on Java, it will run on any operating system where the JVM (Java Virtual Machine) is installed. The Audit4j Core 2.x.x binaries require the JDK (Java Development Kit) version 7.0 or above.
3. Installation
You can download the latest version of Audit4j from the download page, or you can add the repository for Maven, Ant or Gradle. Please refer the download page for more information.
4. Configuration
Even though Audit4j is designed to run with minimum configurations, it still provides various options for customization. Currently, YAML and XML are file formats that are supported.
The Audit4j configuration module uses a set of predefined instructions to load the config file. This allows the user to externalize the configurations from various locations including the environment variable, system variable, classpath lookup, as well as simple file or directory path. Hence, you can work with the same application code in different environments as per the requirement.
The Audit4j configuration lookup algorithm is in the following order;
- 1. Check whether configurations that are directly injected are available in the Context. If available; proceed with the injected configuration. If not, jump to step 7.
- 2. Check if the configuration file or directory path is injected to the context. If available, it scans for a YAML (audit4j.conf.yml/audit4j.conf.yaml) or XML (audit4j.conf.xml) configuration file.
- 3. Scans for the environment variable called "AUDIT4J_CONF_FILE_PATH" in order to locate the configuration file.
- 4. Locate the Java system property variable called “audit4j.conf.file.path” to retrieve the configuration file.
- 5. Searches for the application classpath in order to load the YAML or XML configuration file.
- 6. Check for the user directory (user-dir) for the configuration file. If the file is not available, proceed to step 7.
- 7. If config file is not available in the predefined or detected directory, a YAML configuration file with the default options is automatically created.
Users do not have to create the initial configuration file. If the configuration file is not found, audit4j will automatically create a new configuration file with the default settings in the relevant path, using the above algorithm. It will be easier to make changes around a newly created configuration file.
4.1 Configuration File
Audit4j uses a YAML like markup language in the main configuration file, which will make it easier to read and understand the syntax when compared with a traditional approach (i.e.: XML, JSON, etc). When initializing audt4j, a scan will commence for a configuration file; “audit4j.conf.yml”.
The sample configuration file will look like;
!Configuration # Mandetory
# Configure handlers, One or more handlers must be configured.
handlers:
- !org.audit4j.core.handler.ConsoleAuditHandler {}
- !org.audit4j.core.handler.file.FileAuditHandler {}
# Configure layouts, Either one handler must be configured.
layout: !org.audit4j.core.SecureLayout {}
# Configure meta data.
metaData: !org.audit4j.core.DummyMetaData {}
# Configure additional properties.
properties:
log.file.location: user.dir
4.2 Servlet Context Configurations
For Java web applications, Audit4j supports the loading of configurations from the web application classpath. Optionally it supports the customization of configuration attributes as context-params in a web.xml file.
Servlet context configurations support for Servlet-2.x specification as well as Servlet-3 specification.
- For Servlet spec 2.x
<listener> <listener-class>org.audit4j.core.web.AuditContextListener</listener-class> </listener>
For servlet 3 spec you don’t want to configure in web.xml.
4.3 Spring Configurations
Audit4j supports configurations in the spring application context. Please refer the Spring Integration section under Plugins for more information.
5. Sending audit events
5.1 Calling Audit Manager
The easiest way to integrate Audit4j with your application is by adding the Audit4j core module. Download the instructions here. To send audit records you need to call the Audit Manager.
Java
public void myMethod(String myParam1, Object myParam2) {
String actor = MyApplicationContext.getAuthanticatedUser();
AuditManager manager = AuditManager.getInstance();
manager.audit(new AuditEvent(actor, "myMethod", new Field("myParam1Name", myParam1),
new Field("myParam2Name", myParam2)));
}
Scala
def myMethod(myParam1: String, myParam2: AnyRef) {
val actor = MyApplicationContext.getAuthanticatedUser
val manager = AuditManager.getInstance
manager.audit(new AuditEvent(actor, "myMethod", new Field("myParam1Name", myParam1), new Field("myParam2Name",
myParam2)))
}
Or using Event Builder
Java
String actor = MyApplicationContext.getAuthanticatedUser();
EventBuilder builder = new EventBuilder();
builder.addActor(actor).addAction("myMethod").addField("myParam1Name", myParam1).addField("myParam2Name", myParam2);
AuditManager manager = AuditManager.getInstance();
manager.audit(builder.build());
Scala
val actor = MyApplicationContext.getAuthanticatedUser
val builder = new EventBuilder()
builder.addActor(actor).addAction("myMethod").addField("myParam1Name", myParam1)
.addField("myParam2Name", myParam2)
val manager = AuditManager.getInstance
manager.audit(builder.build())
groovy
5.2 Using annotations
Without writing codes inside the method, audit records can be sent using annotations.
Before using annotation please refer Meta DataSpring section for more details.
Annotations can be used through below plugins.
- “”
- “Weld”
The annotations given below are available with Audit4j.
5.2.1 @Audit
@Audit annotation can be used either in class level or method level. The following options are available in Audit annotation
Example:
@Audit
public void foo(Integer bar){
[su_table]
| @Audit Attribute | Description | Default |
|---|---|---|
| action | Custom action name. | Method name |
| repository | Used to route audit record into different repositories. As an example, for a different audit table. repository is given by the table name. | default |
[/su_table]
5.2.2 @AuditField
@AuditField Annotation is used to mark parameters which will include in the audit event.
[su_table]
| @AuditField Attributes | Description |
|---|---|
| field | Marked field name. |
[/su_table]
5.2.3 @IgnoreAudit
This annotation can be used in below scenarios.
- When The particular class marked for audit using @Audit annotation but specific method should not be audited, This annotation can be added.
Example:
@Audit
public class UserService {
@IgnoreAudit
public void changePassword(String oldPassword, String newPassword) {
// This method will not be audited.
}
}
- Mark method field which should not be audited.
public class UserService {
@Audit
public void login(String username, @IgnoreAudit String password) {
// password parameter will not be audited.
}
}
- Skip auditing method object field.
public class UserService {
@Audit
public void saveUser(User user) {
// password parameter in user object will not be audited.
}
}
public class User {
private String username;
@IgnoreAudit
private String password;
//Getters and Setters
}
5.2.4 @DeIdentify
@DeIdentify can be used to deidentify certain fields. This applies a mask for certain characters with ‘*’. Please read Data De-identification section for more information.
Example:
@Audit
public void saveCreditCardId(@DeIdentify(left=5) int cardId){
Sample input card id: 123456789 Audited record: *****6789
[su_table]
| @DeIdentify Attributes | Description |
|---|---|
| left | De identify left characters. Ex: @DeIdentify(left=3) -> ***456789 |
| right | De identify rightcharacters. Ex: @DeIdentify(right=3) -> 123456*** |
| fromLeft | De identify all characters from the position left. Ex: @DeIdentify(fromLeft=3) -> 123****** |
| fromRight | De identify all characters from the position right. Ex: @DeIdentify(fromRight=3) -> ******789 |
[/su_table]
6 Meta Data Implementation
If you are using annotations to capture audit records, a custom meta-data class should be implemented. Audit4j will not receive certain information using annotations (i.e.: information regarding the logged in user and origin). In such a scenario, Audit4j will add some dummy data into your audit record. This can be carried out easily by implementing org.audit4j.core.MetaData class.
Example for the custom metadata implementation:
import org.audit4j.core.MetaData;
public class MyMetaData implements MetaData{
@Override
public String getActor() {
return MyContext.getAuthanticatedUSer();
}
@Override
public String getOrigin() {
return MyContext.getRemoteAddress();
}
}
Configurations:
[js] !Configuration …
Configure meta data.
metaData: !com.myapp.meta.MyMetaData {} … [/js]
7 Using layouts
Layouts allow you to format individual audit events when it is being stored in the file or console. This will provide out-of-the-box support to encrypt, encode, organize, separate and filter audit information.
Here is an example of a formatted audit event;
08/16/2014 13:52:09|Audit4j User|Origin3|myMethod3==>myParam1Name03 java.lang.String:param103,
7.1 Simple Layout
Simple layout will format audit information by applying special characters (pipe-|,space, comma, arrow - =>> and colon - : ).
Configurations:
[js] !Configuration …
Configure layout
layout: !org.audit4j.core.SimpleLayout{} dateFormat: MM-dd-yyyy HH:mm:ss # Optional. Default will use MM/dd/yyyy HH:mm:ss … [/js]
7.2 Secure layout
Using the secure layout you will be able to encrypt sensitive audit information. If you are using the secure layout you need to add a secure key and salt in the Audit4j configurations as below.
Configurations:
[js] !Configuration …
Configure layout
layout: !org.audit4j.core.SecureLayout{} dateFormat: MM-dd-yyyy HH:mm:ss # Optional. Default will use MM/dd/yyyy HH:mm:ss … [/js]
7.3 Customizable Layout
By using a customizable layout, the output format can be modified using templates. The Audit4j event stream gives you the following variables to build a template for a customizable layout.
${eventDate} - Event created date time ${uuid} - Auto generated uuid ${actor} - Actor ${action} - Action ${origin} - Origin ${fields} - Audit Fields
The operations below can be used as a option looping: ${foreach fields field}${field.name} ${field.type}:${field.value}, ${end}
Conditions: ${if fields == null} No Results ${end}
Example for a template.
| ${eventDate} | ${uuid} | ${actor} | ${action} | ${origin} => ${foreach fields field}${field.name} ${field.type}:${field.value}, ${end} |
The Java Minimal Template Engine (JMTE) is the best template engine we found so far in terms of performance. Thus, all the syntax of JMTE can be applied to Customizable Layouts.
8 Data de-identification
On certain occasions, users will want to audit sensitive information (User email addresses, Credit card numbers, etc). This information should be unidentifiable to third parties, but still at some extent be identifiable to achieve our purposes. This can be achieved using the de-identify feature in the Audit4j platform. De-identification currently supports annotations called “@DeIdentify” which can be used in fields level.
Please see @DeIdentify annotation for more information.
9 Plugins
9.1 Database handler
Database audit handler supports the capturing of audit records with popular databases (MySQL, Oracle, SQL Server, etc.). HSQLDB is embedded in to the database audit handler, but we are not recommending to use it in production systems since sensitive audit records better to store in a different location managed by the user.
9.1.1 Configure database handler
Before starting the configuration you should refer the configuration section in the audit4j documentation.
The following line should be added under the handler: section in the audit4j.conf.yml file.
- !org.audit4j.handler.db.DatabaseAuditHandler{}
After this configuration Audit4j will record any audit events in the embedded database. The database file will automatically be created in the class-path. If you want to configure any other database, please refer the Database Configuration section below.
9.1.2 Configure external databases
If you want to configure any other database, you should disable the one already active. You then need to select the connection type; if it is a single connection, a connection pool or a jndi data source.
connection type parameters as below: Single Connection - single Connection Pool - pooled Jndi Datasource - jndi
Other configurations are usual connection parameters.
[js]
- !org.audit4j.handler.db.DatabaseAuditHandler embeded: false db_connection_type: pooled db_driver: com.mysql.jdbc.Driver db_url: jdbc:mysql://localhost/audit4j db_user: username db_password: password [/js]
9.1.3 Configure jndi datasources
[js]
- !org.audit4j.handler.db.DatabaseAuditHandler embeded: false db_connection_type: jndi db_jndi_datasource: java:comp/env/jdbc/database [/js]
9.1.4 External Data Sources.
External data sources can be directly injected into the audit4j configurations before starting the audit4j server. Noted below is an example for the spring configuration.
@Bean
public DataSource dataSource() {
// Application Datasource configurations.
}
@Bean
public DatabaseAuditHandler databaseHandler() {
DatabaseAuditHandler dbHandler = new DatabaseAuditHandler();
dbHandler.setEmbedded("false");
dbHandler.setDataSource(dataSource());
return dbHandler;
}
9.1.5 Configure Connection Pool.
The Audit4j Database plugin uses Hikari as its default connection pool. The Audit4j DB API exposes the additional configuration options in order to fine-tune the connection pool. The following parameters are supported as optional configuration options.
[su_table]
| Property | Description | Default value |
|---|---|---|
| db_pool_autoCommit | This property controls the default auto-commit behavior of connections returned from the pool | true |
| db_pool_connectionTimeout | Controls the maximum number of milliseconds that a client (that's you) will wait for a connection from the pool | 3000 (30 seconds) |
| db_pool_idleTimeout | controls the maximum amount of time that a connection is allowed to sit idle in the pool | |
| db_pool_maxLifetime | controls the maximum lifetime of a connection in the pool. | |
| db_pool_minimumIdle | This property controls the minimum number of idle connections that HikariCP tries to maintain in the pool. | |
| db_pool_maximumPoolSize | This property controls the maximum size that the pool is allowed to reach, including both idle and in-use connections. | 100 |
[/su_table]
9.1.6 Multiple tables.
An audit stream can be routed to multiple tables in a database handler. This feature can be activated by enabling separate table configurations as below.
[js]
- !org.audit4j.handler.db.DatabaseAuditHandler separate: true [/js]
if you are using annotations events can be tagged for the table name in @audit annotation or simply add tagged information to audit event..
@Audit (tag=”login”)
9.2 File handler
Audit4j in-built file handler will save audit events in files with various configuration options. Configuration:
[js] !Configuration
handlers:
- !org.audit4j.core.handler.file.FileAuditHandler {} [/js]
9.2.2 Auto Archive
9.3 Spring Integration
Audit4j-Spring is a plugin for Audit4j for integration between Spring and Audit4j Core. It will enable the below features in addition to the core functionalities.
- Annotation driven auditing.
- Configurations in the Spring Application Context.
9.3.1 Configurations
Spring integration extension enables the annotation processing using two ways. You can use either one method as you preferred.
- Spring AOP configurations:
[xml]
[/xml]
- Spring Advice configurations: The bean given below should be configured in the spring configuration file.
[xml]
After initiating the bean, it should be injected in to the spring transaction proxy under the particular Service as follows;
[xml]
[/xml]
9.3.1 Audit4j Configuration with spring
The Audit4j-Spring plugin supports out-of-the-box configuration options with spring. The bean shown below should be configured in the spring application context.
[xml]
[/xml]
9.4 Audit4j for CDI spec implementations (Jboss Weld)
[su_label type=”warning”]beeta[/su_label]
9.4.1 Configurations.
Configuration for Jboss Weld:
[xml] <beans …
</beans> [/xml]
9.5 HTTP Integration.
9.5.1 Configurations.
web.xml configuration.
[xml]
[/xml]
9.6 Catalina Plugin for Tomcat, JBoss and Wildfly.
[su_label type=”warning”]beeta[/su_label]
9.6.1 Configurations.
Put audit4j-core-2.3.1-all.jar(or higher) and audit4j-tomcat-2.3.1.jar in the tomcat ‘lib’ directory. Add the following Valve line to Tomcat’s server.xml file. The ‘Engine’ line is used to show context.
[xml]