Dynamically Loading RecyclerView Images in Android

4/27/2019 Updated for Kilo 1.2

As in iOS, Android applications often display thumbnail images alongside other text-based content such as contact names or product descriptions in recycler views. However, the images are not usually delivered with the initial request, but are instead retrieved separately afterward. They are typically downloaded in the background as needed to avoid blocking the UI thread, which would temporarily render the application unresponsive.

For example, consider this sample REST service, which returns a list of simulated photo data:

[
  {
    "albumId": 1,
    "id": 1,
    "title": "accusamus beatae ad facilis cum similique qui sunt",
    "url": "https://via.placeholder.com/600/92c952",
    "thumbnailUrl": "https://via.placeholder.com/150/92c952"
  },
  {
    "albumId": 1,
    "id": 2,
    "title": "reprehenderit est deserunt velit ipsam",
    "url": "https://via.placeholder.com/600/771796",
    "thumbnailUrl": "https://via.placeholder.com/150/771796"
  },
  {
    "albumId": 1,
    "id": 3,
    "title": "officia porro iure quia iusto qui ipsa ut modi",
    "url": "https://via.placeholder.com/600/24f355",
    "thumbnailUrl": "https://via.placeholder.com/150/24f355"
  }, ...
]

Each record contains a photo ID, album ID, and title, as well as URLs for both thumbnail and full-size images; for example:

Example Application

A basic user interface for displaying service results in a recycler view is shown below:

The markup for the main activity simply declares an instance of RecyclerView that will occupy the entire screen:

<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:orientation="vertical"
    android:layout_width="match_parent"
    android:layout_height="match_parent">
    <android.support.v7.widget.RecyclerView android:id="@+id/recyclerView"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:scrollbars="vertical"/>
</LinearLayout>

Item markup is shown below:

<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:layout_margin="16dp">
    <ImageView android:id="@+id/imageView"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"/>
    <TextView android:id="@+id/textView"
        android:layout_width="0dp"
        android:layout_height="wrap_content"
        android:layout_weight="1"
        android:layout_marginLeft="8dp"/>
</LinearLayout>

Photo Class

A class representing the photo data might be defined as follows:

class Photo (map: Map<String, Any>) {
    val id = map["id"] as Int
    val title = map["title"] as String
    val thumbnailUrl = URL(map["thumbnailUrl"] as String)
}

The constructor extracts property values from the map data provided by the service. This data will be used later to retrieve the thumbnails.

View Holder and Adapter Classes

View holder and adapter classes are used to produce and configure individual photo item views. Item data is stored in a list of Photo instances, and previously loaded thumbnail images are stored in a map that associates Bitmap instances with photo IDs:

class MainActivity : AppCompatActivity() {
    // Photo view holder
    inner class PhotoViewHolder(view: View) : RecyclerView.ViewHolder(view) {
        ...
    }

    // Photo adapter
    inner class PhotoAdapter : RecyclerView.Adapter<PhotoViewHolder>() {
        ...
    }

    // Photo list
    var photos: List<Photo>? = null

    // Thumbnail cache
    val photoThumbnails = HashMap<Int, Bitmap>()

    ...
}

Main Activity

When the activity is created, the main view is loaded and the recycler view configured:

override fun onCreate(savedInstanceState: Bundle?) {
    super.onCreate(savedInstanceState)

    setContentView(R.layout.activity_main)

    recyclerView.layoutManager = LinearLayoutManager(this)
    recyclerView.adapter = PhotoAdapter()
}

The photo list is loaded the first time the activity resumes. An instance of WebServiceProxy (provided by the open-source Kilo project) is used to retrieve the data. If the call succeeds, the response data is transformed into a list of Photo objects, and the recycler view is refreshed. Otherwise, an error message is logged:

override fun onResume() {
    super.onResume()

    // Load photo data
    if (photos == null) {
        doInBackground({
            val webServiceProxy = WebServiceProxy("GET", URL("https://jsonplaceholder.typicode.com/photos"))

            val photos = webServiceProxy.invoke { inputStream, _ -> ObjectMapper().readValue(inputStream, List::class.java) }

            photos.map { @Suppress("UNCHECKED_CAST") Photo(it as Map<String, Any>) }
        }) { activity, result ->
            result.onSuccess { value ->
                photos = value

                activity?.recyclerView?.adapter?.notifyDataSetChanged()
            }.onFailure { exception ->
                println(exception.message)
            }
        }
    }
}

Background Task Execution

The photo list is loaded asynchronously using doInBackground(), an extension method added to the Activity class. It provides a lambda-based wrapper around AsyncTask that allows the result handler to safely dereference the activity without leaking memory. Because the activity type is a generic, the callback can access the members of the activity without a cast:

fun <A: Activity, R> A.doInBackground(task: () -> R, resultHandler: (activity: A?, result: Result<R>) -> Unit) {
    BackgroundTask(this, task, resultHandler).execute()
}

class BackgroundTask<A: Activity, R>(activity: A,
    private val task: () -> R,
    private val resultHandler: (activity: A?, result: Result<R>) -> Unit
) : AsyncTask<Unit, Unit, R?>() {
    private val activityReference = WeakReference<A>(activity)

    private var value: R? = null
    private var exception: Exception? = null

    override fun doInBackground(vararg params: Unit?): R? {
        try {
            value = task()
        } catch (exception: Exception) {
            this.exception = exception
        }

        return value
    }

    override fun onPostExecute(value: R?) {
        resultHandler(activityReference.get(), if (exception == null) {
            Result.success(value!!)
        } else {
            Result.failure(exception!!)
        })
    }
}

Item Content

Item content is managed by the view holder and adapter classes. The Photo instance is retrieved from the list and used to configure the item view:

// Photo view holder
inner class PhotoViewHolder(view: View) : RecyclerView.ViewHolder(view) {
    val imageView: ImageView = view.findViewById(R.id.imageView)
    val textView: TextView = view.findViewById(R.id.textView)
}

// Photo adapter
inner class PhotoAdapter : RecyclerView.Adapter<PhotoViewHolder>() {
    override fun onCreateViewHolder(parent: ViewGroup, viewType: Int): PhotoViewHolder {
        return PhotoViewHolder(LayoutInflater.from(parent.context).inflate(R.layout.item_photo, parent, false))
    }

    override fun onBindViewHolder(holder: PhotoViewHolder, position: Int) {
        val photo = photos!![position]

        // Attempt to load image from cache
        val thumbnail = photoThumbnails[photo.id]

        holder.imageView.setImageBitmap(thumbnail)

        if (thumbnail == null) {
            ...
        }

        holder.textView.text = photo.title
    }

    override fun getItemCount(): Int {
        return photos?.size ?: 0
    }
}

If the thumbnail image is already available in the cache, it is used to populate the item’s image view. Otherwise, it is asynchronously loaded from the server and added to the cache as shown below:

if (thumbnail == null) {
    // Request image
    doInBackground({
        val webServiceProxy = WebServiceProxy("GET", photo.thumbnailUrl)

        webServiceProxy.invoke { inputStream, _ -> BitmapFactory.decodeStream(inputStream) }
    }) { activity, result ->
        // Add image to cache and update view holder, if visible
        result.onSuccess { value ->
            photoThumbnails[photo.id] = value

            val viewHolder = activity?.recyclerView?.findViewHolderForAdapterPosition(position) as? PhotoViewHolder

            viewHolder?.imageView?.setImageBitmap(value)
        }
    }
}

If the item is still visible when the image request returns, its image view is updated immediately. Otherwise, it will be updated the next time the item is shown.

More Information

Complete source code for this example can be found here. For more information, see the Kilo README.

Do You Need PUT and PATCH?

Conventional wisdom says that REST APIs should be implemented as follows:

  • GET – read
  • POST – add
  • PUT/PATCH – modify
  • DELETE – remove

This mostly works well. Query parameters can be used to supply arguments for GET and DELETE operations. POSTs can use either URL-encoded or multipart form data, standard encodings supported by nearly all HTTP clients.

However, it doesn’t work quite as well for PUT or PATCH. PUT has no standard encoding, and requires the entire resource to be sent in the payload. PATCH was introduced as a workaround to this limitation, but it also lacks a standard encoding, and is not supported by all clients (notably Java).

However, the POST method can also be used to modify resources. The semantics of POST are less strict than PUT, so it can support partial updates, like PATCH. Further, the same encoding used for creating resources (URL-encoded or multipart) can also be used for updates.

For example:

  • POST /products – add a new resource to the “products” collection using the data specified in the request body
  • POST /products/101 – update the existing resource with ID 101 in the products collection using the (possibly partial) data specified in the request body

This approach works particularly well when resources are backed by relational database tables. An “add” POST maps directly to a SQL INSERT operation, and a “modify” POST translates to a SQL UPDATE. The key/value pairs in the body (whether URL-encoded or multipart) can be mapped directly to the table columns.

The approach also supports bulk inserts and updates. POSTing a URL-encoded payload works well for individual records, but JSON, CSV, or XML could easily be used to add or update multiple records at a time.

So, do you really need PUT and PATCH? Given that POST is more flexible, better supported, and can handle both create and update operations, I’d say no. Please share your thoughts in the comments!

Creating a Simple Web Service using HTTP-RPC and Kotlin

HTTP-RPC is an open-source framework for implementing RESTful and REST-like web services in Java. It is extremely lightweight and requires only a Java runtime environment and a servlet container. The entire framework is distributed as a single JAR file that is about 50KB in size, making it an ideal choice for applications where a minimal footprint is desired.

WebService

HTTP-RPC’s WebService type provides an abstract base class for REST-based web services. It extends the similarly abstract HttpServlet class provided by the servlet API.

Service operations are defined by adding public methods to a concrete service implementation. Methods are invoked by submitting an HTTP request for a path associated with a servlet instance. Arguments are provided either via the query string or in the request body, like an HTML form. WebService converts the request parameters to the expected argument types, invokes the method, and writes the return value to the output stream as JSON.

For example, the following class might be used to implement a simple math service:

@WebServlet(urlPatterns={"/math/*"})
public class MathServlet extends WebService {
    @RequestMethod("GET")
    @ResourcePath("/sum")
    public double getSum(double a, double b) {
        return a + b;
    }
}

The following request would cause the method to be invoked, and the service would return the value 6 in response:

GET /math/sum?a=2&b=4

Kotlin

In addition to Java, HTTP-RPC web services can be implemented in Kotlin, a strongly typed, modern programming language that targets the JVM. Kotlin offers a number of features that make it a compelling alternative to Java for server-side development:

  • Type inference
  • Properties
  • Named method parameters
  • Default argument values
  • Data classes
  • Optionals
  • Extensions

For example, the following service (written in Kotlin) provides some basic information about the host system:

@WebServlet(urlPatterns = ["/system-info/*"], loadOnStartup = 1)
class SystemInfoService : WebService() {
    class SystemInfo(
        val hostName: String,
        val hostAddress: String,
        val availableProcessors: Int,
        val freeMemory: Long,
        val totalMemory: Long
    )

    @RequestMethod("GET")
    fun getSystemInfo(): SystemInfo {
        val localHost = InetAddress.getLocalHost()
        val runtime = Runtime.getRuntime()

        return SystemInfo(
            localHost.hostName,
            localHost.hostAddress,
            runtime.availableProcessors(),
            runtime.freeMemory(),
            runtime.totalMemory()
        )
    }
}

Data returned by the service might look like this:

{
  "hostName": "vm.local",
  "hostAddress": "192.168.1.12",
  "availableProcessors": 4,
  "freeMemory": 222234120,
  "totalMemory": 257949696
}

As with Java-based HTTP-RPC services, API documentation can be accessed by appending “?api” to the service URL:

GET /system-info?api

The response might look something like this:

/system-info

  GET () -> SystemInfo
  

SystemInfo

  {
    hostAddress: string,
    hostName: string,
    availableProcessors: integer,
    freeMemory: long,
    totalMemory: long
  }
  

Summary

Although this is a somewhat contrived example, it demonstrates one of Kotlin’s main advantages: brevity. The SystemInfo data class would have been considerably more verbose in Java. More complex services would see similar benefits.

Complete source code for the examples can be found here. For more information, see the HTTP-RPC README.

Introducing Lima

After nearly four years and over 500 stars on GitHub, I've decided it's time to retire MarkupKit. Despite a respectable level of developer interest, the idea of building an application using XML never seemed to fully resonate with the broader iOS community.

However, even in the absence of a markup-based implementation, the concept of declarative UI is still highly applicable. Today I am happy to introduce Lima, a new Swift-based DSL for constructing iOS and tvOS applications. The project's name comes from the nautical L or Lima flag, representing the first letter of the word "layout":

Lima retains most MarkupKit functionality, and improves on it in a number of ways:

  • Because it is written in Swift, UI code written using Lima is compiled. This means it is validated at build time rather than at run time. The lack of compile-time validation was a major drawback to the markup approach.

  • Further, since it is a Swift-based DSL, developers can finally take advantage of code completion. Although I experimented with a number of different approaches over the years, this is something I was never quite able to get working in XML.

  • Again, because it is written in Swift, Lima code is refactorable. It facilitates better code reuse, and allows developers to employ modern Xcode features like image and color asset literals in UI declarations. Lima also reduces overall file count, since a separate XML document is no longer required.

Converting markup to Lima syntax is straightforward. For example, given this markup:

<LMColumnView spacing="16">
    <UIImageView image="world.png"/>
    <UILabel text="Hello, World!"/>
</LMColumnView>

the Lima equivalent is as follows:

LMColumnView(spacing: 16,
    UIImage(image: UIImage(named: "world.png")),
    UILabel(text: "Hello, World!")
)

It's just as readable, and even slightly more concise, since there's no need for closing tags.

Thanks to everyone who has supported or contributed to MarkupKit. I'm hoping you will find Lima even more useful!

For more information, please see the project README.

You Don’t Need GraphQL

1/19/2019 Updated for HTTP-RPC 6.0

GraphQL is a technology that seems to be getting a lot of attention in the developer community at the moment. Advocates describe it as “a better REST”, claming that it offers several advantages over traditional REST APIs:

  • Single request with nested results vs. multiple separate requests
  • Single endpoint for all requests vs. one endpoint per resource
  • Single evolving version vs. multiple (presumably incompatible) versions

For example, the following GraphQL query might be used to retrieve an employee record from a hypothetical service based on the MySQL “employees” sample database. In addition to the employee number, first name, and last name, the query also requests the employee’s title and salary history:

{
  employee(id: 10004) {
    employeeNumber
    firstName
    lastName
    titles {
      title
      fromDate
      toDate
    }
    salaries {
      salary
      fromDate
      toDate
    }
  }
}

The response might look something like this, with some results omitted for brevity:

{
  "employeeNumber": 10004,
  "firstName": "Chirstian",
  "lastName": "Koblick",
  "titles": [
    {
      "title": "Senior Engineer",
      "fromDate": 817794000000,
      "toDate": 253370782800000
    },
    ...
  ],
  "salaries": [
    {
      "salary": 74057,
      "fromDate": 1006837200000,
      "toDate": 253370782800000
    },
    ...
  ]
}

A RESTful Implementation

The data model for the sample database is shown below:

Employees Sample Database

A typical REST API might provide access to employee, title, and salary resources as follows:

GET /employees/10004
{
  "employeeNumber": 10004,
  "firstName": "Chirstian",
  "lastName": "Koblick"
}
GET /employees/10004/titles
[
  {
    "title": "Senior Engineer",
    "fromDate": 817794000000,
    "toDate": 253370782800000
  },
  ...
]
GET /employees/10004/salaries
[
  {
    "salary": 74057,
    "fromDate": 1006837200000,
    "toDate": 253370782800000
  },
  ...
]

This is indeed more verbose than the GraphQL version. However, there is nothing preventing a REST API from providing a similar interface.

For example, the following service method (implemented using the open-source HTTP-RPC framework) returns the same information as the GraphQL query. As with the GraphQL version, all of the data is obtained with a single request:

@RequestMethod("GET")
@ResourcePath("?:employeeNumber")
public void getEmployee(List<String> details) throws SQLException, IOException {
    String employeeNumber = getKey("employeeNumber");

    Parameters parameters = Parameters.parse("SELECT emp_no AS employeeNumber, "
        + "first_name AS firstName, "
        + "last_name AS lastName "
        + "FROM employees WHERE emp_no = :employeeNumber");

    HashMap<String, Object> arguments = new HashMap<>();

    arguments.put("employeeNumber", employeeNumber);

    try (Connection connection = DriverManager.getConnection(DB_URL);
        PreparedStatement statement = connection.prepareStatement(parameters.getSQL())) {
        parameters.apply(statement, arguments);

        try (ResultSet resultSet = statement.executeQuery()) {
            ResultSetAdapter resultSetAdapter = new ResultSetAdapter(resultSet);

            for (String detail : details) {
                switch (detail) {
                    case "titles": {
                        resultSetAdapter.attach("titles", "SELECT title, "
                            + "from_date AS fromDate, "
                            + "to_date AS toDate "
                            + "FROM titles WHERE emp_no = :employeeNumber");

                        break;
                    }

                    case "salaries": {
                        resultSetAdapter.attach("salaries", "SELECT salary, "
                            + "from_date AS fromDate, "
                            + "to_date AS toDate "
                            + "FROM salaries WHERE emp_no = :employeeNumber");

                        break;
                    }
                }
            }

            getResponse().setContentType("application/json");

            JSONEncoder jsonEncoder = new JSONEncoder();

            jsonEncoder.writeValue(resultSetAdapter.next(), getResponse().getOutputStream());
        }
    } 
}

The initial query retreives the employee’s number, first name, and last name from the “employees” table. Subqueries to return the employee’s salary and title history are optionally attached based on the values provided in the details parameter. Column aliases are used in all of the queries to make the field names more JSON-friendly.

Callers can access the API via a standard HTTP GET request, as shown below:

GET /employees/10004?details=titles&details=salaries
{
  "employeeNumber": 10004,
  "firstName": "Chirstian",
  "lastName": "Koblick",
  "titles": [
    {
      "title": "Senior Engineer",
      "fromDate": 817794000000,
      "toDate": 253370782800000
    },
    ...
  ],
  "salaries": [
    {
      "salary": 74057,
      "fromDate": 1006837200000,
      "toDate": 253370782800000
    },
    ...
  ]
}

Additional Observations

GraphQL advocates tout its single-endpoint model as a major advantage over REST. This capability is not exclusive to GraphQL – it is certainly possible for REST APIs to be implemented using a single endpoint as well. However, such a service would probably become untenable very quickly. A collection of independent endpoints, each of which represent a specific resource or set of resources, will most likely be much more manageable in the long run.

Further, the concept of a single evolving version is not unique to GraphQL. Implementing a successful versioning strategy is difficult, and there are many ways of approaching it. However, there is nothing to preclude a REST service from providing backwards compatibility. It is simply one option among many.

Finally, adopting GraphQL requires services to be completely re-implemented using the GraphQL library. For any non-trivial application, this would most likely be a major undertaking. Additionally, it forces clients to use GraphQL as well, rather than standard HTTP operations such as GET and POST. This means that GraphQL APIs also can’t be tested as easily in a web browser or using command-line utilties such as curl.

So, while there are certainly a number of compelling reasons to consider GraphQL, you don’t actually need to use GraphQL to take advantage of them.

For more information on HTTP-RPC, see the project README.

Introducing Kilo

Kilo is an open-source framework for consuming REST services in iOS or tvOS. It is extremely lightweight and provides a convenient, callback-based interface that makes it easy to interact with remote APIs.

For example, the following code snippet shows how a client application might access a simple service that returns a friendly greeting. The request is executed asynchronously, and the result is printed when the call returns:

webServiceProxy.invoke(.get, path: "/hello") { (result: String?, error: Error?) in
    if let greeting = result {
        print(greeting) // "Hello, World!"
    }
}

The project’s name comes from the nautical K or Kilo flag, which means “I wish to communicate with you”:

This article introduces the Kilo framework and provides an overview of its key features.

WebServiceProxy Class

Kilo is distributed as a universal binary that will run in the iOS simulator as well as on an actual device. The framework contains a single class named WebServiceProxy that is used to issue API requests to the server.

Service proxies are initialized via init(session:serverURL:), which takes the following arguments:

  • session – a URLSession instance that is used to create service requests
  • serverURL – the base URL of the service

A service operation is initiated via one of the following methods:

public func invoke(_ method: Method, path: String,
    arguments: [String: Any] = [:], content: Data? = nil, contentType: String? = nil,
    resultHandler: @escaping (_ result: T?, _ error: Error?) -> Void) -> URLSessionTask? { ... }

public func invoke(_ method: Method, path: String,
    arguments: [String: Any] = [:], content: Data? = nil, contentType: String? = nil,
    resultHandler: @escaping (_ result: T?, _ error: Error?) -> Void) -> URLSessionTask? { ... }

public func invoke(_ method: Method, path: String,
    arguments: [String: Any] = [:], content: Data? = nil, contentType: String? = nil,
    responseHandler: @escaping (_ content: Data, _ contentType: String?) throws -> T?,
    resultHandler: @escaping (_ result: T?, _ error: Error?) -> Void) -> URLSessionTask? { ... }

All three methods accept the following arguments:

  • method – the HTTP method to execute
  • path – the path to the requested resource
  • arguments – a dictionary containing the method arguments as key/value pairs
  • content – an optional Data instance representing the body of the request
  • contentType – an optional string value containing the MIME type of the content
  • resultHandler – a callback that will be invoked upon completion of the method

The first version of the method uses JSONSerialization to decode response data. The second uses JSONDecoder to return a decodable value. The third version accepts an additional responseHandler argument to facilitate decoding of custom response content (for example, a UIImage).

All three methods return an instance of URLSessionTask representing the invocation request. This allows an application to cancel a task, if necessary.

Arguments

Like HTML forms, arguments are submitted either via the query string or in the request body. Arguments for GET, PUT, PATCH, and DELETE requests are always sent in the query string.

POST arguments are typically sent in the request body, and may be submitted as either “application/x-www-form-urlencoded” or “multipart/form-data” (determined via the service proxy’s encoding property). However, if a custom body is specified via the content parameter, POST arguments will be sent in the query string.

Any value that provides a description property may be used as an argument. This property is generally used to convert the argument to its string representation. However, Date instances are automatically converted to a 64-bit integer value representing epoch time (the number of milliseconds that have elapsed since midnight on January 1, 1970).

Additionally, array instances represent multi-value parameters and behave similarly to tags in HTML. Further, when using the multi-part form data encoding, instances of URL represent file uploads and behave similarly to tags in HTML forms. Arrays of URL values operate similarly to tags.

Return Values

The result handler is called upon completion of the operation. If successful, the first argument will contain a deserialized representation of the content returned by the server, and the second argument will be nil. Otherwise, the first argument will be nil, and the second will be populated with an Error instance describing the problem that occurred.

Note that, while service requests are typically processed on a background thread, result handlers are always executed on the application’s main thread. This allows result handlers to update the user interface directly, rather than posting a separate update operation to the main queue.

If the server returns an error response, a localized description of the error will be provided in the localized description of the error parameter. Further, if the error is returned with a content type of “text/plain”, the response body will be returned in the error’s debug description.

Example

The following code snippet demonstrates how the WebServiceProxy class might be used to access the operations of a simple math service:

// Create service proxy
let webServiceProxy = WebServiceProxy(session: URLSession.shared, serverURL: URL(string: "http://localhost:8080")!)

// Get sum of "a" and "b"
webServiceProxy.invoke(.get, path: "/math/sum", arguments: [
    "a": 2,
    "b": 4
]) { (result: Int?, error: Error?) in
    // result is 6
}

// Get sum of all values
webServiceProxy.invoke(.get, path: "/math/sum", arguments: [
    "values": [1, 2, 3, 4]
]) { (result: Int?, error: Error?) in
    // result is 10
}

Additional Information

This article introduced the Kilo framework and provided an overview of its key features. For additional information, see the the project README.

 

Efficiently Transforming JDBC Query Results to JSON

1/19/2019 Updated for HTTP-RPC 6.0

A lot of enterprise data is stored in relational databases and accessed via SQL queries. Many web services are little more than HTTP-based wrappers around such queries.

Unfortunately, transforming query results to JSON so it can be consumed by a client application often involves numerous inefficient steps, such as binding each row to a data object and loading the entire data set into memory before serializing it back to the caller. This type of approach has a negative impact on performance and scalabilty. Each row requires multiple heap allocations and constructor invocations, increasing latency and CPU load. Worse, the caller does not receive a response until the entire data set has been processed.

Further, since each response is loaded entirely into memory, high-volume applications require a large amount of RAM, and can only scale through the addition of more physical hardware. Eventually, the garbage collector has to run, slowing down the entire system.

A much more efficient approach is to stream response data. Instead of copying the query results into an in-memory data structure before sending the response, the web service can write a row of data to the output stream each time a row is read from the result set. This allows a client to begin receiving the data as soon as it is available, significantly reducing latency. Also, because no intermediate data structures are created, CPU and memory load is reduced, allowing each server to handle a higher number of concurrent requests. Finally, because fewer heap allocations are required, the garbage collector needs to run much less frequently, resulting in fewer system pauses.

Introducing HTTP-RPC

HTTP-RPC is an open-source framework for implementing REST services in Java. It is extremely lightweight and requires only a Java runtime environment and a servlet container. The entire framework is distributed as a single JAR file that is less than 70KB in size, making it an ideal choice for applications where a minimal footprint is desired.

WebService

HTTP-RPC’s WebService type provides an abstract base class for REST-based web services. It extends the similarly abstract HttpServlet class provided by the servlet API.

Service operations are defined by adding public methods to a concrete service implementation. Methods are invoked by submitting an HTTP request for a path associated with a servlet instance. Arguments are provided either via the query string or in the request body, like an HTML form. WebService converts the request parameters to the expected argument types, invokes the method, and writes the return value to the output stream as JSON.

The RequestMethod annotation is used to associate a service method with an HTTP verb such as GET or POST. The optional ResourcePath annotation can be used to associate the method with a specific path relative to the servlet. For example, the following class might be used to implement a web service that performs a simple addition operation:

@WebServlet(urlPatterns={"/math/*"})
public class MathServlet extends WebService {
    @RequestMethod("GET")
    @ResourcePath("/sum")
    public double getSum(double a, double b) {
        return a + b;
    }
}

The following request would cause the method to be invoked, and the service would return the value 6 in response:

GET /math/sum?a=2&b=4

JSONEncoder

The JSONEncoder class, which is used internally by WebService to serialize response data, converts return values to their JSON equivalents as follows:

  • CharSequence: string
  • Number: number
  • Boolean: true/false
  • Iterable: array
  • java.util.Map: object

Note that collection types are not required to support random access; iterability is sufficient. This is an important feature, as it allows service implementations to stream result data rather than buffering it in memory before it is written.

ResultSetAdapter

HTTP-RPC’s ResultSetAdapter class implements the Iterable interface and makes each row in a JDBC result set appear as an instance of Map, allowing query results to be efficiently serialized as an array of JSON objects.

For example, consider a web service that returns the result of a SQL query on this table, taken from the MySQL sample database:

CREATE TABLE pet (
    name VARCHAR(20),
    owner VARCHAR(20),
    species VARCHAR(20), 
    sex CHAR(1), 
    birth DATE, 
    death DATE
);

A method to retrieve a list of all pets belonging to a given owner might be implemented as shown below. Note that the example uses HTTP-RPC’s Parameters class to simplify query execution using named parameters rather than positional values. Also note that the method uses JSONEncoder to explicitly write the results to the output stream rather than simply returning the adapter instance, to ensure that the underlying result set is closed and system resources are not leaked:

@RequestMethod("GET")
public void getPets(String owner) throws SQLException, IOException {
    try (Connection connection = DriverManager.getConnection(DB_URL)) {
        Parameters parameters = Parameters.parse("SELECT name, species, sex, birth FROM pet WHERE owner = :owner");

        HashMap<String, Object> arguments = new HashMap<>();

        arguments.put("owner", owner);

        try (PreparedStatement statement = connection.prepareStatement(parameters.getSQL())) {
            parameters.apply(statement, arguments);

            try (ResultSet resultSet = statement.executeQuery()) {
                JSONEncoder jsonEncoder = new JSONEncoder();

                jsonEncoder.writeValue(new ResultSetAdapter(resultSet), getResponse().getOutputStream());
            }
        }
    } 
}

A response produced by the method might look something like this, where each object in the array represents a row from the result set:

[
  {
    "name": "Claws",
    "species": "cat",
    "sex": "m",
    "birth": 763880400000
  },
  {
    "name": "Chirpy",
    "species": "bird",
    "sex": "f",
    "birth": 905486400000
  },
  {
    "name": "Whistler",
    "species": "bird",
    "sex": null,
    "birth": 881643600000
  }
]

With just a few lines of code, query results can be quickly and efficiently returned to the caller, with no intermediate buffering required.

More Information

Complete source code for this example can be found here. For more information, see the project README.