Accessing Nested Data Structures in Java

Enterprise data structures are not usually flat. More often they are hierarchical, nesting one or more levels deep. For example, an account object might contain a customer object, which might contain an address object, and so on.

Such data structures are often returned by web services. A single call to the server requires less code and incurs less network overhead than multiple calls to retrieve the same information. Often, these data structures are returned as JSON and mapped to strongly typed equivalents such as Java beans on the client side. This works well when modeling the complete server response; however, it can be overkill when only a small subset of the returned data is required.

For example, the sample user service at typicode.com returns a collection of records structured like this:

{
  "id": 1,
  "name": "Leanne Graham",
  "username": "Bret",
  "email": "Sincere@april.biz",
  "address": {
    "street": "Kulas Light",
    "suite": "Apt. 556",
    "city": "Gwenborough",
    "zipcode": "92998-3874",
    "geo": {
      "lat": "-37.3159",
      "lng": "81.1496"
    }
  },
  "phone": "1-770-736-8031 x56442",
  "website": "hildegard.org",
  "company": {
    "name": "Romaguera-Crona",
    "catchPhrase": "Multi-layered client-server neural-net",
    "bs": "harness real-time e-markets"
  }
} 

It would be straightforward to map this content to a bean representation. However, if the caller is only interested in the "catchPhrase" values, for example, it would still require the following class definitions at a minimum:

public static class User {
    private Company company;

    public Company getCompany() {
        return company;
    }

    public void setCompany(Company company) {
        this.company = company;
    }
}

public static class Company {
    private String catchPhrase;

    public String getCatchPhrase() {
        return catchPhrase;
    }

    public void setCatchPhrase(String catchPhrase) {
        this.catchPhrase = catchPhrase;
    }
}

Using HTTP-RPC‘s WebServiceProxy class, the following code could then be used to bind the response data to bean instances, extract the catch-phrase values, and convert them to a JSON list:

URL url = new URL("https://jsonplaceholder.typicode.com/users");

List<User> users = WebServiceProxy.get(url).invoke(BeanAdapter.typeOf(List.class, User.class));

List<String> catchPhrases = users.stream()
    .map(user -> user.getCompany().getCatchPhrase())
    .collect(Collectors.toList());

JSONEncoder jsonEncoder = new JSONEncoder();

jsonEncoder.write(catchPhrases, System.out);

For example:

[
  "Multi-layered client-server neural-net",
  "Proactive didactic contingency",
  "Face to face bifurcated interface",
  "Multi-tiered zero tolerance productivity",
  "User-centric fault-tolerant solution",
  "Synchronised bottom-line interface",
  "Configurable multimedia task-force",
  "Implemented secondary concept",
  "Switchable contextually-based project",
  "Centralized empowering task-force"
]

Alternatively, interfaces could be used in place of the bean types to eliminate some of the boilerplate code:

public interface User {
    Company getCompany();
}

public interface Company {
    String getCatchPhrase();
}

The code for extracting the catch-phrases would be identical to the previous example, and the resulting output would be the same:

URL url = new URL("https://jsonplaceholder.typicode.com/users");

List<User> users = WebServiceProxy.get(url).invoke(BeanAdapter.typeOf(List.class, User.class));

List<String> catchPhrases = users.stream()
    .map(user -> user.getCompany().getCatchPhrase())
    .collect(Collectors.toList());

JSONEncoder jsonEncoder = new JSONEncoder();

jsonEncoder.write(catchPhrases, System.out);

A third option would be to deserialize the "raw" JSON data and access the catch-phrases via successive calls to Map#get():

URL url = new URL("https://jsonplaceholder.typicode.com/users");

List<Map<String, Map<String, ?>>> users = WebServiceProxy.get(url).invoke();

List<String> catchPhrases = users.stream()
    .map(user -> (String)user.get("company").get("catchPhrase"))
    .collect(Collectors.toList());

JSONEncoder jsonEncoder = new JSONEncoder();

jsonEncoder.write(catchPhrases, System.out);

This uses less code than the bean or interface approaches, but still requires the declaration of a moderately complex generic, even for this fairly simple case.

A fourth alternative would be to use the valueAt() method of HTTP-RPC’s Collections class to access the nested values by key path:

URL url = new URL("https://jsonplaceholder.typicode.com/users");

List<?> users = WebServiceProxy.get(url).invoke();

List<String> catchPhrases = users.stream()
    .map(user -> (String)Collections.valueAt(user, "company", "catchPhrase"))
    .collect(Collectors.toList());

JSONEncoder jsonEncoder = new JSONEncoder();

jsonEncoder.write(catchPhrases, System.out);

This approach is the least verbose, as it allows the caller to retrieve the desired data directly, without the need for intermediate types or nested generics.

If a caller needs access to most or all of the data returned by a service, then binding to bean or interface types is probably the most practical solution. However, if access to only a targeted subset of nested data is required (e.g. for lightweight transformation or basic validation), then the generic map or valueOf() approach may be preferable.

For more information, see the project README.

Introducing QueryBuilder

Programmatically constructing SQL queries is a common but cumbersome task. For example, consider this query that retrieves information about a hypothetical IoT device:

select Device.*, Building.name as buildingName, Site.name as siteName 
from Device 
join Building on buildingID = Building.id
join Site on siteID = Site.id
where Device.id = :deviceID

Written in plain text, as above, the query is fairly easy to read and understand. However, in Java, the same query might be declared as follows:

String sql = "select Device.*, Building.name as buildingName, Site.name as siteName " 
    + "from Device "
    + "join Building on buildingID = Building.id "
    + "join Site on siteID = Site.id "
    + "where Device.id = :deviceID";

Even in this simple example, readability is negatively impacted by the addition of double quotes and string concatenation operators. Care must be taken to ensure that whitespace is properly managed. Longer, more complex, queries become increasingly difficult to write and maintain.

The new QueryBuilder class introduced in HTTP-RPC 7.5 simplifies the task of writing SQL queries in Java. Using QueryBuilder, the preceding example could be rewritten as follows:

String sql = QueryBuilder.select("Device.*", "Building.name as buildingName", "Site.name as siteName")
    .from("Device")
    .join("Building").on("buildingID = Building.id")
    .join("Site").on("siteID = Site.id")
    .where("Device.id = :deviceID").toString();

Because SQL verbs and clauses are represented as Java methods and nouns (tables, columns, and predicates) by strings, this version is much more readable and easier to maintain. It also scales better as query complexity increases.

A complete example using QueryBuilder along with HTTP-RPC’s Parameters and ResultSetAdapter classes is shown below. Parameters is used to prepare SQL statements for execution using named parameter values rather than indexed arguments. ResultSetAdapter provides access to the contents of a result set via the Iterable and Map interfaces, facilitating direct serialization of the result data to JSON:

String sql = QueryBuilder.select("Device.*", "Building.name as buildingName", "Site.name as siteName")
    .from("Device")
    .join("Building").on("buildingID = Building.id")
    .join("Site").on("siteID = Site.id")
    .where("Device.id = :deviceID").toString();

Parameters parameters = Parameters.parse(sql);

try (PreparedStatement statement = connection.prepareStatement(parameters.getSQL())) {
    parameters.apply(statement, mapOf(
        entry("deviceID", deviceID)
    ));

    try (ResultSetAdapter resultSetAdapter = new ResultSetAdapter(statement.executeQuery())) {
        JSONEncoder jsonEncoder = new JSONEncoder();

        jsonEncoder.write(resultSetAdapter, System.out);
    }
}

For more information, see the project README.

Named Parameters in JDBC Queries

2/15/2021 Updated for HTTP-RPC 7.6

Prepared statements are a common way to execute parameterized queries in JDBC. For example, the following SQL might be used to retrieve a list of all users whose first or last name matches a particular character sequence:

SELECT * FROM user 
WHERE first_name LIKE ? or last_name LIKE ?

Parameter values are supplied at runtime via indexed setter methods defined by the PreparedStatement class:

statement.setString(1, pattern);
statement.setString(2, pattern);

This works fine for simple queries, but it becomes increasingly difficult to manage as the number of parameters grows. It is also redundant – although this query only requires a single argument, two parameter values must be supplied.

The Java Persistence API (JPA) provides a more convenient alternative using named parameters. For example, the above query might be written as follows in JPQL:

SELECT u FROM User u 
WHERE u.firstName LIKE :pattern or u.lastName LIKE :pattern

This is more readable and less verbose, as the caller only needs to provide the value of the “pattern” parameter once. It is also more resilient to changes, as the arguments are not dependent on ordinal position. Unfortunately, it requires a JPA-compliant object-relational mapping (ORM) framework such as Hibernate, a dependency that may not be satisfiable in some situations.

The Parameters class provided by the HTTP-RPC framework brings named parameter support to JDBC. The parse() method of this class is used to create a Parameters instance from a JPA-like SQL query; for example:

SELECT * FROM user 
WHERE first_name LIKE :pattern or last_name LIKE :pattern

It takes a string or reader containing the query text as an argument:

Parameters parameters = Parameters.parse(query);

The getSQL() method of the Parameters class returns the processed query in standard JDBC syntax. This value can be used in a call to the connection’s prepareStatement() method:

PreparedStatement statement = connection.prepareStatement(parameters.getSQL());

Argument values are specified via the apply() method, as shown below (mapOf() is a convenience method provided by the framework for declaratively instantiating maps values, similar to Map.of() in Java 9):

parameters.apply(statement, mapOf(
entry("pattern", pattern)
));

Once applied, the query can be executed:

ResultSet resultSet = statement.executeQuery();

Note that the Parameters class is not limited to queries; it can also be used for updates.

A complete example can be found here. It is a simple REST service that allows a caller to search a database of pets by owner name.

See the project README for more information.

Regular Expression to Validate a Comma-Separated List of Email Addresses

Recently, I needed to create a regular expression to validate the format of a comma-separated list of email addresses. Just thought I’d share the result in case it is of use to anyone. The pattern I came up with for matching a single address is as follows:

[\w-]+(\.[\w-]+)*@[\w-]+(\.[\w-]+)+

Note that I didn’t attempt to handle every possible combination of valid characters – only what I’d consider the most common ones.

The pattern is simply duplicated for the comma-separated list. Here’s an example of applying the pattern in Java:

Pattern addressListPattern = Pattern.compile(String.format("%1$s(,\\s*%1$s)*",
    "[\\w-]+(\\.[\\w-]+)*@[\\w-]+(\\.[\\w-]+)+"),
    Pattern.UNICODE_CHARACTER_CLASS);

System.out.println(addressListPattern.matcher("xyz").matches()); // false
System.out.println(addressListPattern.matcher("foo@bar.com").matches()); // true
System.out.println(addressListPattern.matcher("foo@bar.com, xyz").matches()); // false
System.out.println(addressListPattern.matcher("foo@b-ar.com, f.oo@b-ar.co.uk").matches()); // true
System.out.println(addressListPattern.matcher("føo@b-ar.com, f-oo@b-ar.co.uk").matches()); // true

Note that Pattern.UNICODE_CHARACTER_CLASS is not necessary (or supported) in Android, which supports Unicode character matching by default.

Hope it helps!