Documentation

  1. Introduction.
  2. Requirements.
  3. Installation.
  4. Configuration.
  5. Sending Audit Events.
    1. Calling Audit Manager.
    2. Using Annotations.
      1. @Audit
      2. @AuditField
      3. @DeIdentify
      4. @IgnoreAudit
  6. Implementing Meta Data class.
  7. Using Layouts.
    1. Simple Layout.
    2. Secure Layout.
    3. Customizable Layout.
  8. Data de-identification.
  9. Plugins.
    1. Database Handler.
      1. Configure Database Handler.
      2. Configure External Databases.
      3. Configure jndi Data Sources.
      4. External Data Sources.
      5. Configure Connection Pool.
      6. Multiple tables.
    2. File Handler.
      1. Auto Archive.
    3. Spring Integration.
      1. Configurations.
      2. Audit4j Configuration with spring.
    4. Audit4j for CDI spec implementations (Jboss Weld).
      1. Configurations.
    5. HTTP Integration.
      1. Configurations.
    6. Catalina Plugin for Tomcat, JBoss and Wildfly.
      1. Configurations.
    7. Glassfish Plugin.
      1. Configurations.
    8. Jetty Plugin.
      1. Configurations.
  10. Custom Event Filters.
  11. Monitoring And Management.
  12. Unit testing with Audit4j.
  13. Performance Tuning.
  14. Plugin developments.

 

1. Introduction

Audit4j is an open source auditing framework which is a full stack application auditing solution for java enterprise projects. Refer the FAQ (frequently asked question) section for an overview of the application and the audit4j framework.

Audit4j is Open Source and available under the Apache 2 License.

Download from http://audit4j.org/downloads.

Code examples are prepared, which are included in a set of demo projects in order to demonstrate the usage of Audit4j and its plugins. For further information, please refer the following URL in order to access the project on Github.

http://github.com/audit4j/audit4j-demo

This documentation undergoes constant modifications (i.e.: added features, improvements, etc).

This document serves as the guide to the Audit4j framework. It explains the integration of the application, its configuration, optimization, security and usage of the plugin.

 

2. requirements

Audit4j has been tested on a common distributions of Linux, Windows and Mac OS. Even though Audit4j is entirely built on Java, it will run on any operating system where the JVM (Java Virtual Machine) is installed. The Audit4j Core 2.x.x binaries require the JDK (Java Development Kit) version 7.0 or above.

 

3. Installation

You can download the latest version of Audit4j from the download page, or you can add the repository for Maven, Ant or Gradle. Please refer the download page for more information.

 

4. Configuration

Even though Audit4j is designed to run with minimum configurations, it still provides various options for customization. Currently, YAML and XML are file formats that are supported.

The Audit4j configuration module uses a set of predefined instructions to load the config file. This allows the user to externalize the configurations from various locations including the environment variable, system variable, classpath lookup, as well as simple file or directory path. Hence, you can work with the same application code in different environments as per the requirement.

The Audit4j configuration lookup algorithm is in the following order;

Users do not have to create the initial configuration file. If the configuration file is not found, audit4j will automatically create a new configuration file with the default settings in the relevant path, using the above algorithm. It will be easier to make changes around a newly created configuration file.

 

4.1 Configuration File

Audit4j uses a YAML like markup language in the main configuration file, which will make it easier to read and understand the syntax when compared with a traditional approach (i.e.: XML, JSON, etc). When initializing audt4j, a scan will commence for a configuration file; “audit4j.conf.yml”.

The sample configuration file will look like;

!Configuration # Mandetory

# Configure handlers, One or more handlers must be configured.

handlers:
- !org.audit4j.core.handler.ConsoleAuditHandler {}
- !org.audit4j.core.handler.file.FileAuditHandler {}

# Configure layouts, Either one handler must be configured.

layout: !org.audit4j.core.SecureLayout {}

# Configure meta data.

metaData: !org.audit4j.core.DummyMetaData {}

# Configure additional properties.
properties:
log.file.location: user.dir

4.2 Servlet Context Configurations

For Java web applications, Audit4j supports the loading of configurations from the web application classpath. Optionally it supports the customization of configuration attributes as context-params in a web.xml file.

Servlet context configurations support for Servlet-2.x specification as well as Servlet-3 specification.

  1. For Servlet spec 2.x
    <listener>
    <listener-class>org.audit4j.core.web.AuditContextListener</listener-class>
    </listener>
    

For servlet 3 spec you don’t want to configure in web.xml.

4.3 Spring Configurations

Audit4j supports configurations in the spring application context. Please refer the Spring Integration section under Plugins for more information.

5. Sending audit events

5.1 Calling Audit Manager

The easiest way to integrate Audit4j with your application is by adding the Audit4j core module. Download the instructions here. To send audit records you need to call the Audit Manager.

Java

public void myMethod(String myParam1, Object myParam2) {
String actor = MyApplicationContext.getAuthanticatedUser();
AuditManager manager = AuditManager.getInstance();
manager.audit(new AuditEvent(actor, "myMethod", new Field("myParam1Name", myParam1),
new Field("myParam2Name", myParam2)));
}

Scala

def myMethod(myParam1: String, myParam2: AnyRef) {
val actor = MyApplicationContext.getAuthanticatedUser
val manager = AuditManager.getInstance
manager.audit(new AuditEvent(actor, "myMethod", new Field("myParam1Name", myParam1), new Field("myParam2Name",
myParam2)))
}

Or using Event Builder

Java

String actor = MyApplicationContext.getAuthanticatedUser();
EventBuilder builder = new EventBuilder();
builder.addActor(actor).addAction("myMethod").addField("myParam1Name", myParam1).addField("myParam2Name", myParam2);
AuditManager manager = AuditManager.getInstance();
manager.audit(builder.build());

Scala

val actor = MyApplicationContext.getAuthanticatedUser
val builder = new EventBuilder()
builder.addActor(actor).addAction("myMethod").addField("myParam1Name", myParam1)
.addField("myParam2Name", myParam2)
val manager = AuditManager.getInstance
manager.audit(builder.build())

groovy

 

5.2 Using annotations

Without writing codes inside the method, audit records can be sent using annotations.

Before using annotation please refer Meta DataSpring section for more details.

Annotations can be used through below plugins.

  1. Weld

The annotations given below are available with Audit4j.

5.2.1 @Audit

@Audit annotation can be used either in class level or method level. The following options are available in Audit annotation

Example:

@Audit
public void foo(Integer bar){

 

[su_table]

@Audit Attribute Description Default
action Custom action name. Method name
repository Used to route audit record into different repositories. As an example, for a different audit table. repository is given by the table name. default

[/su_table]

 

5.2.2 @AuditField

@AuditField Annotation is used to mark parameters which will include in the audit event.

[su_table]

@AuditField Attributes Description
field Marked field name.

[/su_table]

 

5.2.3 @IgnoreAudit

This annotation can be used in below scenarios.

  1. When The particular class marked for audit using @Audit annotation but specific method should not be audited, This annotation can be added.

Example:

@Audit
public class UserService {
@IgnoreAudit
public void changePassword(String oldPassword, String newPassword) {
// This method will not be audited.
}
}
  1. Mark method field which should not be audited.
public class UserService {
@Audit
public void login(String username, @IgnoreAudit String password) {
// password parameter will not be audited.
}
}
  1. Skip auditing method object field.
public class UserService {
@Audit
public void saveUser(User user) {
// password parameter in user object will not be audited.
}
}


public class User {

private String username;
@IgnoreAudit
private String password;

//Getters and Setters
}

 

5.2.4 @DeIdentify

@DeIdentify can be used to deidentify certain fields. This applies a mask for certain characters with ‘*’. Please read Data De-identification section for more information.

Example:

@Audit
public void saveCreditCardId(@DeIdentify(left=5) int cardId){

Sample input card id: 123456789 Audited record: *****6789

[su_table]

@DeIdentify Attributes Description
left De identify left characters. Ex: @DeIdentify(left=3) -> ***456789
right De identify rightcharacters. Ex: @DeIdentify(right=3) -> 123456***
fromLeft De identify all characters from the position left. Ex: @DeIdentify(fromLeft=3) -> 123******
fromRight De identify all characters from the position right. Ex: @DeIdentify(fromRight=3) -> ******789

[/su_table]

 

6 Meta Data Implementation

If you are using annotations to capture audit records, a custom meta-data class should be implemented. Audit4j will not receive certain information using annotations (i.e.: information regarding the logged in user and origin). In such a scenario, Audit4j will add some dummy data into your audit record. This can be carried out easily by implementing org.audit4j.core.MetaData class.

Example for the custom metadata implementation:

import org.audit4j.core.MetaData;
public class MyMetaData implements MetaData{
@Override
public String getActor() {
return MyContext.getAuthanticatedUSer();
}

    @Override
    public String getOrigin() {
        return MyContext.getRemoteAddress();
    }
}

Configurations:

[js] !Configuration …

Configure meta data.

metaData: !com.myapp.meta.MyMetaData {} … [/js]

 

7 Using layouts

Layouts allow you to format individual audit events when it is being stored in the file or console. This will provide out-of-the-box support to encrypt, encode, organize, separate and filter audit information.

Here is an example of a formatted audit event;

08/16/2014 13:52:09|Audit4j User|Origin3|myMethod3==>myParam1Name03 java.lang.String:param103,

 

7.1 Simple Layout

Simple layout will format audit information by applying special characters (pipe-|,space, comma, arrow - =>> and colon - : ).

Configurations:

[js] !Configuration …

Configure layout

layout: !org.audit4j.core.SimpleLayout{} dateFormat: MM-dd-yyyy HH:mm:ss # Optional. Default will use MM/dd/yyyy HH:mm:ss … [/js]

 

7.2 Secure layout

Using the secure layout you will be able to encrypt sensitive audit information. If you are using the secure layout you need to add a secure key and salt in the Audit4j configurations as below.

Configurations:

[js] !Configuration …

Configure layout

layout: !org.audit4j.core.SecureLayout{} dateFormat: MM-dd-yyyy HH:mm:ss # Optional. Default will use MM/dd/yyyy HH:mm:ss … [/js]

7.3 Customizable Layout

By using a customizable layout, the output format can be modified using templates. The Audit4j event stream gives you the following variables to build a template for a customizable layout.

${eventDate} - Event created date time ${uuid} - Auto generated uuid ${actor} - Actor ${action} - Action ${origin} - Origin ${fields} - Audit Fields

The operations below can be used as a option looping: ${foreach fields field}${field.name} ${field.type}:${field.value}, ${end}

Conditions: ${if fields == null} No Results ${end}

Example for a template.

${eventDate} ${uuid} ${actor} ${action} ${origin} => ${foreach fields field}${field.name} ${field.type}:${field.value}, ${end}

The Java Minimal Template Engine (JMTE) is the best template engine we found so far in terms of performance. Thus, all the syntax of JMTE can be applied to Customizable Layouts.

 

8 Data de-identification

On certain occasions, users will want to audit sensitive information (User email addresses, Credit card numbers, etc). This information should be unidentifiable to third parties, but still at some extent be identifiable to achieve our purposes. This can be achieved using the de-identify feature in the Audit4j platform. De-identification currently supports annotations called “@DeIdentify” which can be used in fields level.

Please see @DeIdentify annotation for more information.

 

9 Plugins

 

9.1 Database handler

Database audit handler supports the capturing of audit records with popular databases (MySQL, Oracle, SQL Server, etc.). HSQLDB is embedded in to the database audit handler, but we are not recommending to use it in production systems since sensitive audit records better to store in a different location managed by the user.

 

9.1.1 Configure database handler

Before starting the configuration you should refer the configuration section in the audit4j documentation.

The following line should be added under the handler: section in the audit4j.conf.yml file.

- !org.audit4j.handler.db.DatabaseAuditHandler{}

After this configuration Audit4j will record any audit events in the embedded database. The database file will automatically be created in the class-path. If you want to configure any other database, please refer the Database Configuration section below.

 

9.1.2 Configure external databases

If you want to configure any other database, you should disable the one already active. You then need to select the connection type; if it is a single connection, a connection pool or a jndi data source.

connection type parameters as below: Single Connection - single Connection Pool - pooled Jndi Datasource - jndi

Other configurations are usual connection parameters.

[js]

 

9.1.3 Configure jndi datasources

 

[js]

 

9.1.4 External Data Sources.

External data sources can be directly injected into the audit4j configurations before starting the audit4j server. Noted below is an example for the spring configuration.

@Bean
public DataSource dataSource() {
// Application Datasource configurations.
}

@Bean
public DatabaseAuditHandler databaseHandler() {
DatabaseAuditHandler dbHandler = new DatabaseAuditHandler();
dbHandler.setEmbedded("false");
dbHandler.setDataSource(dataSource());
return dbHandler;
}

 

9.1.5 Configure Connection Pool.

The Audit4j Database plugin uses Hikari as its default connection pool. The Audit4j DB API exposes the additional configuration options in order to fine-tune the connection pool. The following parameters are supported as optional configuration options.

[su_table]

Property Description Default value
db_pool_autoCommit This property controls the default auto-commit behavior of connections returned from the pool true
db_pool_connectionTimeout Controls the maximum number of milliseconds that a client (that's you) will wait for a connection from the pool 3000 (30 seconds)
db_pool_idleTimeout controls the maximum amount of time that a connection is allowed to sit idle in the pool
db_pool_maxLifetime controls the maximum lifetime of a connection in the pool.
db_pool_minimumIdle This property controls the minimum number of idle connections that HikariCP tries to maintain in the pool.
db_pool_maximumPoolSize This property controls the maximum size that the pool is allowed to reach, including both idle and in-use connections. 100

[/su_table]

9.1.6 Multiple tables.

An audit stream can be routed to multiple tables in a database handler. This feature can be activated by enabling separate table configurations as below.

[js]

if you are using annotations events can be tagged for the table name in @audit annotation or simply add tagged information to audit event..

@Audit (tag=login)

9.2 File handler

Audit4j in-built file handler will save audit events in files with various configuration options. Configuration:

[js] !Configuration

handlers:

9.2.2 Auto Archive

 

9.3 Spring Integration

Audit4j-Spring is a plugin for Audit4j for integration between Spring and Audit4j Core. It will enable the below features in addition to the core functionalities.

 

9.3.1 Configurations

Spring integration extension enables the annotation processing using two ways. You can use either one method as you preferred.

  1. Spring AOP configurations:

[xml]

[/xml]

  1. Spring Advice configurations: The bean given below should be configured in the spring configuration file.

[xml][/xml]

After initiating the bean, it should be injected in to the spring transaction proxy under the particular Service as follows;

[xml]

auditAdvice

[/xml]

 

9.3.1 Audit4j Configuration with spring

The Audit4j-Spring plugin supports out-of-the-box configuration options with spring. The bean shown below should be configured in the spring application context.

[xml]

[/xml]

 

9.4 Audit4j for CDI spec implementations (Jboss Weld)

[su_label type=”warning”]beeta[/su_label]

9.4.1 Configurations.

Configuration for Jboss Weld:

[xml] <beans …

org.audit4j.intregration.cdi.AuditInterceptor

</beans> [/xml]

 

9.5 HTTP Integration.

9.5.1 Configurations.

web.xml configuration.

[xml]

auditFilter org.audit4j.intregration.http.AuditFilter auditFilter /*

[/xml]

 

9.6 Catalina Plugin for Tomcat, JBoss and Wildfly.

[su_label type=”warning”]beeta[/su_label]

9.6.1 Configurations.

Put audit4j-core-2.3.1-all.jar(or higher) and audit4j-tomcat-2.3.1.jar in the tomcat ‘lib’ directory. Add the following Valve line to Tomcat’s server.xml file. The ‘Engine’ line is used to show context.

[xml]

[/xml]  

9.7 Glassfish Plugin

[su_label type="warning"]beeta[/su_label]

9.7.1 Configurations.

For glashfish 4.0+ [xml] <sun-web-app ...> ... </sun-web-app> [/xml]  

9.8 Jetty Plugin

[su_label type="warning"]beeta[/su_label]

9.8.1 Configurations.

For Jetty 8,9+ Server Mode: [xml] ... [/xml] Embedded Mode: ```java Server server=new Server(8080); ... Audit4jJettyHandler wrapper = new Audit4jJettyHandler(); server.setHandler(wrapper); server.start(); } ``` For Jetty 6.1.5 Server Mode: Put audit4j-core-2.3.1.jar(or higher) and audit4j-tomcat-2.3.1.jar in the jetty 'lib/etc' directory. [xml] ... [/xml]  

10 Custom Event Filters.

Custom filters are used to screen audit events. Custom filter should implement AuditEventFilter interface. Sample custom event filter implantation, This filter will not accepts the events with action "DummyAction". ```java public class CustomFilter implements AuditEventFilter { @Override public boolean accepts(AuditEvent event) { if (event.getAction().equals("DummyAction")) { return false; } return true; } ``` Optionally filters provides the filter query language to simplify the development. Below filter will filter events with action, "Action1" or "Action2". ```java public class CustomFilterWithQuery implements AuditEventFilter{ @Override public boolean accepts(AuditEvent event) { if (query.from(event).with("action").eq("Action1").or().eq("Action2").evaluate()) { return false; } return true; } ```  

11 Monitoring And Management

Audit4j can be monitored and managed using Java Management Extensions (JMX). In other words, JMX can be used to monitor the Audit4j server status and manage basic server operations including starting, stopping, enabling and disabling. JMX can be enabled via the configuration due to the fact that the JMX setting is disabled by default. Use below configurations to enable JMX. [js] jmx: !org.audit4j.core.jmx.JMXConfig {} [/js]  

12 Unit testing with audit4j

Audit4j supports the use of different settings other than production settings for unit tests in Audit4j integrated applications. Different configurations for test scope will be supported by Audit4j. As an example, if you need to point to a different database for audit trails in your unit tests, Audit4j provides an API which enables the use of different configurations in unit test classes. Configurations should be added to a setup method in your unit test class before initiating Audit4j. ```java @Before public void before() { AuditManager.startWithConfiguration("/home/audit4j/audit4j-test.conf.yml"); } ``` Write your test cases as usual. ```java @Test public void test(){ ...... } ```   Make sure you shut down the Audit4j context in teardown method to clear the audit4j resources. ```java @After public void after() { AuditManager.getInstance().shutdown(); } ```  

13 Performance Tuning

Audit4j supports limited options to improve performance in your application. These options are not enabled by default because they are highly depend on your applications’ nature and the system environment.

13.1 Metadata lookup in async mode

By default metadata lookup is setted to sync mode and this can be changed to async mode by using below configuration. [js] commands: -metadata=async [/js] Warning: The metadata lookup will not work if your application framework doesn’t allow invoking metadata information asynchronously.  

14 Plugin Development