Documentation

Contents.

  1. 1 Introduction.
  2. 2 Requirements.
  3. 3 Installation.
  4. 4 Configuration.
  5. 5 Sending Audit Events.
    1. 5.1 Calling Audit Manager.
    2. 5.2 Using Annotations.
      1. 5.2.1 @Audit
      2. 5.2.2 @AuditField
      3. 5.2.3 @DeIdentify
      4. 5.2.4 @IgnoreAudit
  6. 6 Implementing Meta Data class.
  7. 7 Using Layouts.
    1. 7.1 Simple Layout.
    2. 7.2 Secure Layout.
    3. 7.3 Customizable Layout.
  8. 8 Data de-identification.
  9. 9 Plugins.
    1. 9.1 Database Handler.
      1. 9.1.1 Configure Database Handler.
      2. 9.1.2 Configure External Databases.
      3. 9.1.3 Configure jndi Data Sources.
      4. 9.1.4 External Data Sources.
      5. 9.1.5 Configure Connection Pool.
      6. 9.1.6 Multiple tables.
    2. 9.2 File Handler.
      1. 9.2.1 Auto Archive.
    3. 9.3 Spring Integration.
      1. 9.3.1 Configurations.
      2. 9.3.2 Audit4j Configuration with spring.
    4. 9.4 Audit4j for CDI spec implementations (Jboss Weld).
      1. 9.4.1 Configurations.
    5. 9.5 HTTP Integration.
      1. 9.5.1 Configurations.
    6. 9.6 Catalina Plugin for Tomcat, JBoss and Wildfly.
      1. 9.6.1 Configurations.
    7. 9.7 Glassfish Plugin.
      1. 9.7.1 Configurations.
    8. 9.8 Jetty Plugin.
      1. 9.8.1 Configurations.
  10. 10 Custom Event Filters.
  11. 11 Monitoring And Management.
  12. 12 Unit testing with Audit4j.
  13. 13 Performance Tuning.
  14. 14 Plugin developments.

 

1. Introduction

Audit4j is an open source auditing framework which is a full stack application auditing solution for java enterprise projects. Refer the FAQ (frequently asked question) section for an overview of the application and the audit4j framework.

Audit4j is Open Source and available under the Apache 2 License.

Download from http://audit4j.org/downloads.

Code examples are prepared, which are included in a set of demo projects in order to demonstrate the usage of Audit4j and its plugins. For further information, please refer the following URL in order to access the project on Github.

http://github.com/audit4j/audit4j-demo

This documentation undergoes constant modifications (i.e.: added features, improvements, etc).

This document serves as the guide to the Audit4j framework. It explains the integration of the application, its configuration, optimization, security and usage of the plugin.

 

2. requirements

Audit4j has been tested on a common distributions of Linux, Windows and Mac OS. Even though Audit4j is entirely built on Java, it will run on any operating system where the JVM (Java Virtual Machine) is installed. The Audit4j Core 2.x.x binaries require the JDK (Java Development Kit) version 7.0 or above.

 

3. Installation

You can download the latest version of Audit4j from the download page, or you can add the repository for Maven, Ant or Gradle. Please refer the download page for more information.

 

4. Configuration

Even though Audit4j is designed to run with minimum configurations, it still provides various options for customization. Currently, YAML and XML are file formats that are supported.

The Audit4j configuration module uses a set of predefined instructions to load the config file. This allows the user to externalize the configurations from various locations including the environment variable, system variable, classpath lookup, as well as simple file or directory path. Hence, you can work with the same application code in
different environments as per the requirement.

The Audit4j configuration lookup algorithm is in the following order;

  • 1. Check whether configurations that are directly injected are available in the Context. If available; proceed with the injected configuration. If not, jump to step 7.
  • 2. Check if the configuration file or directory path is injected to the context. If available, it scans for a YAML (audit4j.conf.yml/audit4j.conf.yaml) or XML (audit4j.conf.xml) configuration file.
  • 3. Scans for the environment variable called “AUDIT4J_CONF_FILE_PATH” in order to locate the configuration file.
  • 4. Locate the Java system property variable called “audit4j.conf.file.path” to retrieve the configuration file.
  • 5. Searches for the application classpath in order to load the YAML or XML configuration file.
  • 6. Check for the user directory (user-dir) for the configuration file. If the file is not available, proceed to step 7.
  • 7. If config file is not available in the predefined or detected directory, a YAML configuration file with the default options is automatically created.

Users do not have to create the initial configuration file. If the configuration file is not found, audit4j will automatically create a new configuration file with the default settings in the relevant path, using the above algorithm. It will be easier to make changes around a newly created configuration file.

 

4.1 Configuration File

Audit4j uses a YAML like markup language in the main configuration file, which will make it easier to read and understand the syntax when compared with a traditional approach (i.e.: XML, JSON, etc). When initializing audt4j, a scan will commence for a configuration file; “audit4j.conf.yml”.

The sample configuration file will look like;

!Configuration # Mandetory

# Configure handlers, One or more handlers must be configured.
handlers:
- !org.audit4j.core.handler.ConsoleAuditHandler {}
- !org.audit4j.core.handler.file.FileAuditHandler {}

# Configure handlers, Either one handler must be configured.
layout: !org.audit4j.core.SecureLayout {}

# Configure meta data.
metaData: !org.audit4j.core.DummyMetaData {}

# Configure additional properties.
properties:
   log.file.location: user.dir

4.2 Servlet Context Configurations

For Java web applications, Audit4j supports the loading of configurations from the web application classpath. Optionally it supports the customization of configuration attributes as context-params in a web.xml file.

Servlet context configurations support for Servlet-2.x specification as well as Servlet-3 specification.

1. For Servlet spec 2.x

  <listener>
    <listener-class>org.audit4j.core.web.AuditContextListener</listener-class>
  </listener>

For servlet 3 spec you don’t want to configure in web.xml.

4.3 Spring Configurations

Audit4j supports configurations in the spring application context. Please refer the Spring Integration section under Plugins for more information.

5. Sending audit events

5.1 Calling Audit Manager

The easiest way to integrate Audit4j with your application is by adding the Audit4j core module. Download the instructions here. To send audit records you need to call the Audit Manager.

JavaScalaGroovy
public void myMethod(String myParam1, Object myParam2) {
     String actor = MyApplicationContext.getAuthanticatedUser();
     AuditManager manager = AuditManager.getInstance();
     manager.audit(new AuditEvent(actor, "myMethod", new Field("myParam1Name", myParam1),
              new Field("myParam2Name", myParam2)));
 }
def myMethod(myParam1: String, myParam2: AnyRef) {
    val actor = MyApplicationContext.getAuthanticatedUser
    val manager = AuditManager.getInstance
    manager.audit(new AuditEvent(actor, "myMethod", new Field("myParam1Name", myParam1), new Field("myParam2Name",
      myParam2)))
  }

Or using Event Builder

JavaScalaGroovy
String actor = MyApplicationContext.getAuthanticatedUser();
EventBuilder builder = new EventBuilder();
builder.addActor(actor).addAction("myMethod").addField("myParam1Name", myParam1).addField("myParam2Name", myParam2);
AuditManager manager = AuditManager.getInstance();
manager.audit(builder.build());
val actor = MyApplicationContext.getAuthanticatedUser
    val builder = new EventBuilder()
    builder.addActor(actor).addAction("myMethod").addField("myParam1Name", myParam1)
      .addField("myParam2Name", myParam2)
    val manager = AuditManager.getInstance
    manager.audit(builder.build())

 

5.2 Using annotations

Without writing codes inside the method, audit records can be sent using annotations.

Before using annotation please refer Meta DataSpring section for more details.

Annotations can be used through below plugins.
1. “
2. “Weld

The annotations given below are available with Audit4j.

5.2.1 @Audit

@Audit annotation can be used either in class level or method level. The following options are available in Audit annotation

Example:

@Audit
public void foo(Integer bar){

 

@Audit Attribute Description Default
action Custom action name. Method name
repository Used to route audit record into different repositories. As an example, for a different audit table. repository is given by the table name. default

 

5.2.2 @AuditField

@AuditField Annotation is used to mark parameters which will include in the audit event.

@AuditField
Attributes
Description
field Marked field name.

 

5.2.3 @IgnoreAudit

This annotation can be used in below scenarios.

1. When The particular class marked for audit using @Audit annotation but specific method should not be audited, This annotation can be added.

Example:

@Audit
public class UserService {
    @IgnoreAudit
    public void changePassword(String oldPassword, String newPassword) {
        // This method will not be audited.
    }
}

2. Mark method field which should not be audited.

public class UserService {
    @Audit
    public void login(String username, @IgnoreAudit String password) {
        // password parameter will not be audited.
    }
}

3. Skip auditing method object field.

public class UserService {
    @Audit
    public void saveUser(User user) {
        // password parameter in user object will not be audited.
    }
}


public class User {

private String username;
@IgnoreAudit
private String password;

//Getters and Setters
}

 

5.2.4 @DeIdentify

@DeIdentify can be used to deidentify certain fields. This applies a mask for certain characters with ‘*’. Please read Data De-identification section for more information.

Example:

@Audit
public void saveCreditCardId(@DeIdentify(left=5) int cardId){

Sample input card id: 123456789
Audited record: *****6789

@DeIdentify
Attributes
Description
left De identify left characters. Ex: @DeIdentify(left=3) -> ***456789
right De identify rightcharacters. Ex: @DeIdentify(right=3) -> 123456***
fromLeft De identify all characters from the position left. Ex: @DeIdentify(fromLeft=3) -> 123******
fromRight De identify all characters from the position right. Ex: @DeIdentify(fromRight=3) -> ******789

 

6 Meta Data Implementation

If you are using annotations to capture audit records, a custom meta-data class should be implemented. Audit4j will not receive certain information using annotations (i.e.: information regarding the logged in user and origin). In such a scenario, Audit4j will add some dummy data into your audit record. This can be carried out easily by implementing org.audit4j.core.MetaData class.

Example for the custom metadata implementation:

import org.audit4j.core.MetaData;
public class MyMetaData implements MetaData{
    @Override
    public String getActor() {
        return MyContext.getAuthanticatedUSer();
    }

    @Override
    public String getOrigin() {
        return MyContext.getRemoteAddress();
    }
 }

Configurations:

!Configuration
...
# Configure meta data.
metaData: !com.myapp.meta.MyMetaData {}
...

 

7 Using layouts

Layouts allow you to format individual audit events when it is being stored in the file or console. This will provide out-of-the-box support to encrypt, encode, organize, separate and filter audit information.

Here is an example of a formatted audit event;

08/16/2014 13:52:09|Audit4j User|Origin3|myMethod3==>myParam1Name03 java.lang.String:param103,

 

7.1 Simple Layout

Simple layout will format audit information by applying special characters (pipe-|,space, comma, arrow – =>> and colon – : ).

Configurations:

!Configuration
...
# Configure layout
layout: !org.audit4j.core.SimpleLayout{}
  dateFormat: MM-dd-yyyy HH:mm:ss # Optional. Default will use MM/dd/yyyy HH:mm:ss
...

 

7.2 Secure layout

Using the secure layout you will be able to encrypt sensitive audit information.
If you are using the secure layout you need to add a secure key and salt in the Audit4j configurations as below.

Configurations:

!Configuration
...
# Configure layout
layout: !org.audit4j.core.SecureLayout{}
  dateFormat: MM-dd-yyyy HH:mm:ss # Optional. Default will use MM/dd/yyyy HH:mm:ss
...

7.3 Customizable Layout

By using a customizable layout, the output format can be modified using templates. The Audit4j event stream gives you the following variables to build a template for a customizable layout.

${eventDate} – Event created date time
${uuid} – Auto generated uuid
${actor} – Actor
${action} – Action
${origin} – Origin
${fields} – Audit Fields

The operations below can be used as a option
looping:
${foreach fields field}${field.name} ${field.type}:${field.value}, ${end}

Conditions:
${if fields == null} No Results ${end}

Example for a template.

${eventDate}|${uuid}|${actor}|${action}|${origin} => ${foreach fields field}${field.name} ${field.type}:${field.value}, ${end}

The Java Minimal Template Engine (JMTE) is the best template engine we found so far in terms of performance. Thus, all the syntax of JMTE can be applied to Customizable Layouts.

 

8 Data de-identification

On certain occasions, users will want to audit sensitive information (User email addresses, Credit card numbers, etc). This information should be unidentifiable to third parties, but still at some extent be identifiable to achieve our purposes. This can be achieved using the de-identify feature in the Audit4j platform.
De-identification currently supports annotations called “@DeIdentify” which can be used in fields level.

Please see @DeIdentify annotation for more information.

 

9 Plugins

 

9.1 Database handler

Database audit handler supports the capturing of audit records with popular databases (MySQL, Oracle, SQL Server, etc.). HSQLDB is embedded in to the database audit handler, but we are not recommending to use it in production systems since sensitive audit records better to store in a different location managed by the user.

 

9.1.1 Configure database handler

Before starting the configuration you should refer the configuration section in the audit4j documentation.

The following line should be added under the handler: section in the audit4j.conf.yml file.

- !org.audit4j.handler.db.DatabaseAuditHandler{}

After this configuration Audit4j will record any audit events in the embedded database. The database file will automatically be created in the class-path. If you want to configure any other database, please refer the Database Configuration section below.

 

9.1.2 Configure external databases

If you want to configure any other database, you should disable the one already active. You then need to select the connection type; if it is a single connection, a connection pool or a jndi data source.

connection type parameters as below:
Single Connection – single
Connection Pool – pooled
Jndi Datasource – jndi

Other configurations are usual connection parameters.

- !org.audit4j.handler.db.DatabaseAuditHandler
embeded: false
db_connection_type: pooled
db_driver: com.mysql.jdbc.Driver
db_url: jdbc:mysql://localhost/audit4j
db_user: username
db_password: password

 

9.1.3 Configure jndi datasources

 

- !org.audit4j.handler.db.DatabaseAuditHandler
  embeded: false
  db_connection_type: jndi
  db_jndi_datasource: java:comp/env/jdbc/database

 

9.1.4 External Data Sources.

External data sources can be directly injected into the audit4j configurations before starting the audit4j server. Noted below is an example for the spring configuration.

@Bean
public DataSource dataSource() {
   // Application Datasource configurations.
}

@Bean
public DatabaseAuditHandler databaseHandler() {
    DatabaseAuditHandler dbHandler = new DatabaseAuditHandler();
        dbHandler.setEmbedded("false");
        dbHandler.setDataSource(dataSource());
        return dbHandler;
}

 

9.1.5 Configure Connection Pool.

The Audit4j Database plugin uses Hikari as its default connection pool. The Audit4j DB API exposes the additional configuration options in order to fine-tune the connection pool. The following parameters are supported as optional configuration options.

Property Description Default value
db_pool_autoCommit This property controls the default auto-commit behavior of connections returned from the pool true
db_pool_connectionTimeout Controls the maximum number of milliseconds that a client (that’s you) will wait for a connection from the pool 3000 (30 seconds)
db_pool_idleTimeout controls the maximum amount of time that a connection is allowed to sit idle in the pool
db_pool_maxLifetime controls the maximum lifetime of a connection in the pool.
db_pool_minimumIdle This property controls the minimum number of idle connections that HikariCP tries to maintain in the pool.
db_pool_maximumPoolSize This property controls the maximum size that the pool is allowed to reach, including both idle and in-use connections. 100

9.1.6 Multiple tables.

An audit stream can be routed to multiple tables in a database handler. This feature can be activated by enabling separate table configurations as below.

- !org.audit4j.handler.db.DatabaseAuditHandler
separate: true

if you are using annotations events can be tagged for the table name in @audit annotation or simply add tagged information to audit event..

@Audit (tag=”login”)

9.2 File handler

Audit4j in-built file handler will save audit events in files with various configuration options.
Configuration:

!Configuration

handlers:
- !org.audit4j.core.handler.file.FileAuditHandler {}

9.2.2 Auto Archive

 

9.3 Spring Integration

Audit4j-Spring is a plugin for Audit4j for integration between Spring and Audit4j Core. It will enable the below features in addition to the core functionalities.

  • Annotation driven auditing.
  • Configurations in the Spring Application Context.

 

9.3.1 Configurations

Spring integration extension enables the annotation processing using two ways. You can use either one method as you preferred.

1. Spring AOP configurations:

<bean id="auditAspect" class="org.audit4j.integration.spring.AuditAspect" />

<aop:aspectj-autoproxy>
   <aop:include name="auditAspect" />
</aop:aspectj-autoproxy>

2. Spring Advice configurations:
The bean given below should be configured in the spring configuration file.

<bean id="auditAdvice" class="org.audit4j.integration.spring.AuditAdvice" />

After initiating the bean, it should be injected in to the spring transaction proxy under the particular Service as follows;

<bean id="serviceClassImpl" class="com.myproject.ServiceImpl">
    <!-- properties here -->
</bean>

<bean id="serviceImplProxy"
         class="org.springframework.aop.framework.ProxyFactoryBean">
    <property name="target" ref="serviceClassImpl" />
        <property name="interceptorNames">
        <list>
            <value>auditAdvice</value>
        </list>
     </property>
</bean>

 

9.3.1 Audit4j Configuration with spring

The Audit4j-Spring plugin supports out-of-the-box configuration options with spring. The bean shown below should be configured in the spring application context.

<bean id="layout" class="org.audit4j.core.SimpleLayout"></bean>
<bean id="metaData" class="com.myproject.util.AuditMetaData"></bean>
<bean id="consoleAuditHandler" class="org.audit4j.core.handler.ConsoleAuditHandler"></bean>
<bean id="databaseAuditHandler" class="org.audit4j.handler.db.DatabaseAuditHandler">
    <property name="embedded" value="false"></property>
    <property name="db_connection_type" value="jndi"></property>
    <property name="db_jndi_datasource" value="java:comp/env/jdbc/database"></property>
</bean>
<bean id="auditconfig" class="org.audit4j.integration.spring.SpringAudit4jConfig">
    <property name="layout" ref="layout"></property>
    <property name="metaData" ref="metaData"></property>
    <property name="handlers">
        <list>
            <ref bean="consoleAuditHandler"/>
            <ref bean="databaseAuditHandler"/>
        </list>
    </property>
</bean>

 

9.4 Audit4j for CDI spec implementations (Jboss Weld)

beeta

9.4.1 Configurations.

Configuration for Jboss Weld:

<beans
 ...
    <interceptors>
    <class>org.audit4j.intregration.cdi.AuditInterceptor</class>
    </interceptors>
</beans>

 

9.5 HTTP Integration.

9.5.1 Configurations.

web.xml configuration.

<filter>
    <filter-name>auditFilter</filter-name>
    <filter-class>org.audit4j.intregration.http.AuditFilter</filter-class>
</filter>

<filter-mapping>
   <filter-name>auditFilter</filter-name>
   <url-pattern>/*</url-pattern>
</filter-mapping>

 

9.6 Catalina Plugin for Tomcat, JBoss and Wildfly.

beeta

9.6.1 Configurations.

Put audit4j-core-2.3.1-all.jar(or higher) and audit4j-tomcat-2.3.1.jar in the tomcat ‘lib’ directory.
Add the following Valve line to Tomcat’s server.xml file. The ‘Engine’ line is used to show context.

<Engine name="Catalina" defaultHost="localhost">
 <Valve className="org.audit4j.intregration.tomcat.Audit4jTomcatValve"/>

 

9.7 Glassfish Plugin

beeta

9.7.1 Configurations.

For glashfish 4.0+

<sun-web-app ...>
   ...
   <property name="auditValue" value="org.audit4j.intregration.glassfish.Audit4jGlashfishValue"/>
</sun-web-app>

 

9.8 Jetty Plugin

beeta

9.8.1 Configurations.

For Jetty 8,9+
Server Mode:

<Set name="handler">
   ...
   <Item>
      <New id="auditHandler" class="org.audit4j.intregration.jetty.Audit4jJettyHandler"/>
   </Item>
</Set>

Embedded Mode:

 Server server=new Server(8080);
 ...
 Audit4jJettyHandler wrapper =  new Audit4jJettyHandler();
 server.setHandler(wrapper);
 server.start();
}

For Jetty 6.1.5
Server Mode:
Put audit4j-core-2.3.1.jar(or higher) and audit4j-tomcat-2.3.1.jar in the jetty ‘lib/etc’ directory.

<Set name="handler">
...
  <Item>
    <New id="auditHandler" class="org.audit4j.intregration.jetty.Audit4jJetty6Handler"/>
  </Item>
</Set>

 

10 Custom Event Filters.

Custom filters are used to screen audit events. Custom filter should implement AuditEventFilter interface.

Sample custom event filter implantation, This filter will not accepts the events with action “DummyAction”.

public class CustomFilter implements AuditEventFilter {
@Override
public boolean accepts(AuditEvent event) {
    if (event.getAction().equals("DummyAction")) {
        return false;
    }
    return true;
}

Optionally filters provides the filter query language to simplify the development. Below filter will filter events with action, “Action1″ or “Action2″.

public class CustomFilterWithQuery implements AuditEventFilter{
@Override
public boolean accepts(AuditEvent event) {
    if (query.from(event).with("action").eq("Action1").or().eq("Action2").evaluate()) {
        return false;
   }
   return true;
}

 

11 Monitoring And Management

Audit4j can be monitored and managed using Java Management Extensions (JMX). In other words, JMX can be used to monitor the Audit4j server status and manage basic server operations including starting, stopping, enabling and disabling. JMX can be enabled via the configuration due to the fact that the JMX setting is disabled by default. Use below configurations to enable JMX.

jmx: !org.audit4j.core.jmx.JMXConfig {}

 

12 Unit testing with audit4j

Audit4j supports the use of different settings other than production settings for unit tests in Audit4j integrated applications. Different configurations for test scope will be supported by Audit4j. As an example, if you need to point to a different database for audit trails in your unit tests, Audit4j provides an API which enables the use of different configurations in unit test classes.

Configurations should be added to a setup method in your unit test class before initiating Audit4j.

@Before
public void before() {
   AuditManager.startWithConfiguration("/home/audit4j/audit4j-test.conf.yml");
}

Write your test cases as usual.

@Test
public void test(){
......
}

 

Make sure you shut down the Audit4j context in teardown method to clear the audit4j resources.

@After
public void after() {
   AuditManager.getInstance().shutdown();
}

 

13 Performance Tuning

Audit4j supports limited options to improve performance in your application. These options are not enabled by default because they are highly depend on your applications’ nature and the system environment.

13.1 Metadata lookup in async mode

By default metadata lookup is setted to sync mode and this can be changed to async mode by using below configuration.

commands: -metadata=async

Warning: The metadata lookup will not work if your application framework doesn’t allow invoking metadata information asynchronously.

 

14 Plugin Development