Request Validation and Filtering by Flags – Redesign and Refactoring

General
In the previous posts I started describing a validation / filtering framework we’re building.
While showing the code, I am trying to show clean code, test orientation and code evolution.
It has some agility in the process; We know the end requirements, but the exact details are evolving over time.

During the development we have changed the code to be more general as we saw some patterns in it.
The code evolved as the flow evolved as well.

The flow as we now understand it
Here’s a diagram of the flow we’ll implement

Request Sequence

Request Sequence

The Pattern
At each step of the sequence (validation, filtering, action), we recognized the same pattern:

  1. We have specific implementations (filters, validations)
  2. We have an engine that wraps up the specific implementations
  3. We need to map the implementations by flag, and upon request’s flags, select the appropriate implementations.
  4. We need to have a class that calls the mapper and then the engine

A diagram showing the pattern

The Pattern

The Pattern

Source Code
In order to show some of the evolution of the code, and how refactoring changed it, I added tags in GitHub after major changes.

Code Examples
Let’s see what came up from the mapper pattern.

public interface MapperByFlag<T> {
  List<T> getOperations(Request request);
}
public abstract class AbstractMapperByFlag<T> implements MapperByFlag<T> {
  private List<T> defaultOperations;
  private Map<String, List<T>> mapOfOperations;

  public AbstractMapperByFlag(List<T> defaultOperations, Map<String, List<T>> mapOfOperations) {
    this.defaultOperations = defaultOperations;
    this.mapOfOperations = mapOfOperations;
  }

  @Override
  public final List<T> getOperations(Request request) {
    Set<T> selectedFilters = Sets.newHashSet(defaultOperations);
    Set<String> flags = request.getFlags();
    for (String flag : flags) {
      if (mapOfOperations.containsKey(flag)) {
        selectedFilters.addAll(mapOfOperations.get(flag));
      }
    }
    return Lists.newArrayList(selectedFilters);
  }
}
  public RequestValidationByFlagMapper(List<RequestValidation> defaultValidations,
    map<String, List<RequestValidation>> mapOfValidations) {
    super(defaultValidations, mapOfValidations);
  }

  public ItemFiltersByFlagMapper(List<Filter> defaultFilters, Map<String, List<Filter>> mapOfFilters) {
    super(defaultFilters, mapOfFilters);
  }

I created a test for the abstract class, to show the flow itself.
The tests of the implementations use Java Reflection to verify that the correct injected parameters are sent to the super.
I am showing the imports here as well. To have some reference for the static imports, mockito and hamcrest packages and classes.

import static org.hamcrest.Matchers.containsInAnyOrder;
import static org.junit.Assert.assertThat;
import static org.mockito.Mockito.when;

import java.util.List;
import java.util.Map;

import org.eyal.requestvalidation.model.Request;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.Mock;
import org.mockito.runners.MockitoJUnitRunner;

import com.google.common.collect.ImmutableMap;
import com.google.common.collect.Lists;
import com.google.common.collect.Sets;

@RunWith(MockitoJUnitRunner.class)
public class AbstractMapperByFlagTest {
	private final static String FLAG_1 = "flag 1";
	private final static String FLAG_2 = "flag 2";

	@Mock
	private Request request;

	private String defaultOperation1 = "defaultOperation1";
	private String defaultOperation2 = "defaultOperation2";
	private String mapOperation11 = "mapOperation11";
	private String mapOperation12 = "mapOperation12";
	private String mapOperation23 = "mapOperation23";

	private MapperByFlag<String> mapper;

	@Before
	public void setup() {
		List<String> defaults = Lists.newArrayList(defaultOperation1, defaultOperation2);
		Map<String, List<String>> mapped = ImmutableMap.<String, List<String>> builder()
		        .put(FLAG_1, Lists.newArrayList(mapOperation11, mapOperation12))
		        .put(FLAG_2, Lists.newArrayList(mapOperation23, mapOperation11)).build();
		mapper = new AbstractMapperByFlag<String>(defaults, mapped) {
		};
	}

	@Test
	public void whenRequestDoesNotHaveFlagsShouldReturnDefaultFiltersOnly() {
		when(request.getFlags()).thenReturn(Sets.<String> newHashSet());

		List<String> filters = mapper.getOperations(request);
		assertThat(filters, containsInAnyOrder(defaultOperation1, defaultOperation2));
	}

	@Test
	public void whenRequestHasFlagsNotInMappingShouldReturnDefaultFiltersOnly() {
		when(request.getFlags()).thenReturn(Sets.<String> newHashSet("un-mapped-flag"));
		List<String> filters = mapper.getOperations(request);
		assertThat(filters, containsInAnyOrder(defaultOperation1, defaultOperation2));
	}
	
	@Test
	public void whenRequestHasOneFlagShouldReturnWithDefaultAndMappedFilters() {
		when(request.getFlags()).thenReturn(Sets.<String> newHashSet(FLAG_1));
		List<String> filters = mapper.getOperations(request);
		assertThat(filters, containsInAnyOrder(mapOperation12, defaultOperation1, mapOperation11, defaultOperation2));
	}
	
	@Test
	public void whenRequestHasTwoFlagsShouldReturnWithDefaultAndMappedFiltersWithoutDuplications() {
		when(request.getFlags()).thenReturn(Sets.<String> newHashSet(FLAG_1, FLAG_2));
		List<String> filters = mapper.getOperations(request);
		assertThat(filters, containsInAnyOrder(mapOperation12, defaultOperation1, mapOperation11, defaultOperation2, mapOperation23));
	}
}
@RunWith(MockitoJUnitRunner.class)
public class RequestValidationByFlagMapperTest {

	@Mock
	private List<RequestValidation> defaultValidations;
    
	@Mock
	private Map<String, List<RequestValidation>> mapOfValidations;

	@InjectMocks
	private RequestValidationByFlagMapper mapper;

	@SuppressWarnings("unchecked")
    @Test
	public void verifyParameters() throws NoSuchFieldException, SecurityException, IllegalArgumentException,
	        IllegalAccessException {
		Field defaultOperationsField = AbstractMapperByFlag.class.getDeclaredField("defaultOperations");
		defaultOperationsField.setAccessible(true);
        List<RequestValidation> actualFilters = (List<RequestValidation>) defaultOperationsField.get(mapper);
		assertThat(actualFilters, sameInstance(defaultValidations));

		Field mapOfFiltersField = AbstractMapperByFlag.class.getDeclaredField("mapOfOperations");
		mapOfFiltersField.setAccessible(true);
		Map<String, List<RequestValidation>> actualMapOfFilters = (Map<String, List<RequestValidation>>) mapOfFiltersField.get(mapper);
		assertThat(actualMapOfFilters, sameInstance(mapOfValidations));
	}
}

To Do
There are other classes that might be candidate for refactoring of some sort.
RequestFlowValidation and RequestFilter are similar.
And
RequestValidationsEngineImpl and FiltersEngine

To Do 2
Create a Matcher for the reflection part.

Code
As always, all the code can be found at:

A Tag for this post: all-components-in

Conclusion
The infrastructure is almost done.
During this time we are also implementing actual classes for the flow (validations, filters, actions).
These are not covered in the posts, nor in GitHub.
The infrastructure will be wired to a service we have using Spring.
This will be explained in future posts.

Request Validation and Filtering by Flags – Introduction

General

We are working on a service that should accept some kind of request.

The request has List of Items. In the response we need to tell the client whether the request is valid and also some information about each item: is it valid or not. If it’s valid, it will be persisted. If it’s not, it should be filtered out. So the response can have information of how many items are valid (and sent to be persisted) and list of information of the filtered out items.

The request has another metadata in it. It has collection (set) of flags. The filtering and validation is based on the flags of the request. So basically one request may be validated and filtered differently than the other, based on the flags of each request.

We might have general validations / filters that need to be applied to any request, whatever flags it has.

Request Validation and Filtering High level design

Design

Flags Mapping

We’ll hold a mapping of flag-to-filters, and flag-to-validation.

Request

Has flags and items.

Components

Filter, Filter-Engine, Flags-Mapper

Development Approach

Bottom Up

We have a basic request already, as the service is up and running, but we don’t have any infrastructure for flags, flag-mapping, validation and filtering.

We’ll work bottom up; create the mechanism for filtering, enhance the request and then wire it up using Spring.

Coding

I’ll try to show the code using tests, and the development using some kind of TDD approach.

I am using eclipse’s EclEmma for coverage.

General

By looking at the code, you can see usage of JUnit, Mockito, Hamcrest, Google-Guava.

You can also see small classes, and interface development approach.

Source Code

https://github.com/eyalgo/request-validation

Recommended Books

I have a list of books, which I highly recommend.
Each book taught me something different.

It all begun years ago, when I went into interviewing process for my second work place.
I was a junior Java developer, a coder. I didn’t have much experience and more importantly, I did not have a mentor or someone who would direct me. I learned on my own, after a CS Java course. Java 1.4 just came.

One of my first interviewers was a great mentor. We met for an hour (probably). I don’t remember the company.  I don’t remember the job position. I don’t remember his name.
But I DO remember a few things he asked me.
He asked me if I know what TDD was. He asked me about XP.
He also recommended a book: Effective Java by Joshua Bloch

He didn’t even know what a great gift he gave me.

So I went on and bought Effective Java, 1st edition. And TDD by Kent Beck.
That was my first step towards being craftsman.

Effective Java and Refactoring
These two books look as they are not entirely related.
However, both of these books thought me a-lot about design and patterns.
I started to understand how to write code using patterns (Refactoring), and how to do it in Java (Effective).
These books gave me the grounds for best practice in Java and Design Patterns and OOD.

Test Driven Development
I can’t say enough about this book.
At first, I really didn’t understand what it was all about.
But it was part of XP !! (which I didn’t understand as well).
The TDD was left on the shelf until I was ready for it.

Clean Code and The Pragmatic Programmer
Should I say more?
If you haven’t read both, stop everything and go to read.
They are MUST for anyone who wants to be craftsman and takes his / her profession seriously.
These books are also lots of fun to read. Especially the Pragmatic book.

The Clean Coder
If you want to take the next step of being a professional, read it.
I was sometimes frustrated while reading it. I thought to myself how can pass all of this material to my teammates…

Dependency Injection
Somewhat not related, but as I see it, if you don’t use DI, you can’t write clean, testable code.
If you can’t write clean, testable code, you are missing the point of craftsmanship.
The book covers some injectors frameworks, but also describe what is it all about.

Below is a table with the books I have mentioned.

One last remark,
This list does not contain the only books I read.
During the years I have read more technical / professional books, but these made the most difference for me.

Name Author(s) ISBN
Effective Java Joshua Bloch 978-032-135-668-0
Test-Driven Development Kent Beck 978-032-114-653-3
Refactoring Martin Fowler 978-020-148-567-7
Dependency Injection Dhanji R. Prasanna 978-193-398-855-9
Clean Code Robert C. Martin 978-013-235-088-4
The Clean Coder Robert C. Martin 978-013-708-107-3
The Pragmatic Programmer Andrew Hunt , David Thomas 978-020-161-622-4

Coding Exercise Introduction

As part of my job, I do a-lot of architectural designing, OOD, clean code, TDD and everything that thrives to be craftsmanship work.

However, I don’t get to have many problems such as tree traverse, BFS, DFS, lists etc.
We can name these kind of problems as CS1 and CS2 courses problems.
I also don‘t have the opportunity to learn new languages. We‘re writing in Java and there is no reason at the office to start learning a new language. At least not for business purpose.

But, as a professional developer, I want to constantly exercise, sharpen and improve my skills.
So I took upon myself a small project:

  1. Do some basic coding that I usually don’t do
  2. Learn a new language

As for task #1, I already wrote some Java code to problems I thought of, and will try to add more during the weeks to come.

As for task #2, I decides to start learning Ruby. Why Ruby? No particular reason. It’s different from Java and good in the market.
Once I get comfortable with Ruby, my plan is to write the problems in Ruby.

[EDIT]
The code is in GitHub
See Why at: Why GitHub
[EDIT]

Some of the code has nice written tests, and some, sadly to say, I just played around, it is not REALLY, AUTOMATICALLY testes. This is something that must be fixed as well.

These are the problems I already written:

  • Factorial
  • Fibonacci
  • Reverse a list
  • Anagram
  • Palindrome
  • BFS tree traverse

Code at GitHub: https://github.com/eyalgo/brainers-java

Getting Started with Google Guava – Book Review

I recently got my hands (my kindle) on the book: Getting Started with Google Guava by Bill Bejeck.

I love reading technical books and always hope to learn new stuff. As an extensive user of the Guava library, I was really intrigued to see what I was missing from this library and how I could improve the usage of it.

I will not go over it chapter by chapter with explanations, as anyone can check the TOC and see the details of what this book covers. Instead, I will try to give my own impression.

The book covers all aspects of the Guava library. For each aspect, the author shows the most used implementation and mentions other ones.

In nearly every chapter, I was introduced to some gems that immediately went into our own codebase when I started refactoring. That was FUN. And I saw code improvements instantly.

I really enjoyed reading the code examples with the extensive usage of JUnit as showcases for the behavior of the various classes. It’s a great way of showing what the library does. And as a side effect, it shows developers how a test is used as the specs of the code.

It seems that the author was very meticulous in writing clean and testable code. Two areas, which I think are, well, the most important for being a professional developer (a craftsman).

I think that this book is great for both newbies and experienced Guava users.
I think it is also great for developers who want to have some kind of knowledge on how to write clean and better code.