Efficiently Producing and Consuming CSV in Java using HTTP-RPC

In a previous article, I discussed how the open-source HTTP-RPC framework can be used to efficiently transform JDBC query results into JSON. However, while JSON is an extremely common and well-supported data format, it may in some cases be preferable to return a CSV document instead. Because field keys are specified only at the beginning of a CSV document rather than being duplicated for every record, CSV generally requires less bandwidth than JSON. Additionally, consumers can begin processing CSV as soon as the first record arrives, rather than waiting for the entire document to download.

The previous article demonstrated how HTTP-RPC's ResultSetAdapter and JSONEncoder classes could be used to easily generate a JSON response from a SQL query on this table, taken from the MySQL sample database:

CREATE TABLE pet (
    name VARCHAR(20),
    owner VARCHAR(20),
    species VARCHAR(20), 
    sex CHAR(1), 
    birth DATE, 
    death DATE
);

The following service method returns the same results encoded as CSV. The only difference in this version is the use of HTTP-RPC's CSVEncoder class instead of JSONEncoder:

@RequestMethod("GET")
public void getPets(String owner) throws SQLException, IOException {
    try (Connection connection = DriverManager.getConnection(DB_URL)) {
        Parameters parameters = Parameters.parse("SELECT name, species, sex, birth FROM pet WHERE owner = :owner");

        parameters.put("owner", owner);

        try (PreparedStatement statement = connection.prepareStatement(parameters.getSQL())) {
            parameters.apply(statement);

            try (ResultSet resultSet = statement.executeQuery()) {
                CSVEncoder csvEncoder = new CSVEncoder();
                
                csvEncoder.writeValue(new ResultSetAdapter(resultSet), getResponse().getOutputStream());
            }
        }
    } finally {
        getResponse().flushBuffer();
    }
}

The response might look something like this, where each record in the document represents a row from the result set (dates are represented using epoch time):

"name","species","sex","birth"
"Claws","cat","m",763880400000
"Chirpy","bird","f",905486400000
"Whistler","bird",,881643600000

Consuming a CSV Response

On the other end, HTTP-RPC's CSVDecoder class can be used to efficiently process the contents of a CSV document. Rather than reading the entire payload into memory and returning the data as a random-access list of map values, CSVDecoder returns a cursor over the records in the document. This allows a client to read records essentially as they are being produced, reducing memory consumption and improving throughput.

For example, the following code could be used to consume the response generated by the web service:

WebServiceProxy webServiceProxy = new WebServiceProxy("GET", new URL("http://localhost:8080/httprpc-test/pets"));

webServiceProxy.getArguments().put("owner", "Gwen");
webServiceProxy.getArguments().put("format", "csv");

webServiceProxy.invoke((inputStream, contentType) -> {
    CSVDecoder csvDecoder = new CSVDecoder();

    CSVDecoder.Cursor pets = csvDecoder.readValues(inputStream);

    for (Map<String, String> pet : pets) {
        System.out.println(String.format("%s is a %s", pet.get("name"), pet.get("species")));
    }

    return null;
});

This code produces the following output:

Claws is a cat
Chirpy is a bird
Whistler is a bird

Additionally, the adapt() method of the CSVDecoder.Cursor class can be used to facilitate typed iteration of CSV data. This method produces an Iterable sequence of values of a given type representing the rows in the document. The returned adapter uses dynamic proxy invocation to map properties declared by the interface to map values in the cursor. A single proxy instance is used for all rows to minimize heap allocation.

For example, the following interface might be used to model the pet records shown above:

public interface Pet {
    public String getName();
    public String getOwner();
    public String getSpecies();
    public String getSex();
    public Date getBirth();
}

This code uses adapt() to create an iterable sequence of Pet values from the CSV response:

CSVDecoder csvDecoder = new CSVDecoder();

CSVDecoder.Cursor pets = csvDecoder.readValues(inputStream);

for (Pet pet : pets.adapt(Pet.class)) {
    System.out.println(String.format("%s is a %s", pet.getName(), pet.getSpecies()));
}

The output is identical to the previous example; however, in this case, the fields of each record are retrieved in a type-safe manner (a feature whose value becomes much more obvious when handling CSV documents containing numeric or boolean values, for example).

More Information

This article provided an example of how HTTP-RPC's CSVEncoder and CSVDecoder classes can be used to efficiently stream and process JDBC query results as CSV. For more information, see the project README.

Introducing Kilo

Kilo is an open-source framework for consuming REST services in iOS or tvOS. It is extremely lightweight and provides a convenient, callback-based interface that makes it easy to interact with remote APIs.

For example, the following code snippet shows how a client application might access a simple service that returns a friendly greeting. The request is executed asynchronously, and the result is printed when the call returns:

webServiceProxy.invoke(.get, path: "/hello") { (result: String?, error: Error?) in
    if let greeting = result {
        print(greeting) // "Hello, World!"
    }
}

The project’s name comes from the nautical K or Kilo flag, which means “I wish to communicate with you”:

This article introduces the Kilo framework and provides an overview of its key features.

WebServiceProxy Class

Kilo is distributed as a universal binary that will run in the iOS simulator as well as on an actual device. The framework contains a single class named WebServiceProxy that is used to issue API requests to the server.

Service proxies are initialized via init(session:serverURL:), which takes the following arguments:

  • session – a URLSession instance that is used to create service requests
  • serverURL – the base URL of the service

A service operation is initiated via one of the following methods:

public func invoke(_ method: Method, path: String,
    arguments: [String: Any] = [:], content: Data? = nil, contentType: String? = nil,
    resultHandler: @escaping (_ result: T?, _ error: Error?) -> Void) -> URLSessionTask? { ... }

public func invoke(_ method: Method, path: String,
    arguments: [String: Any] = [:], content: Data? = nil, contentType: String? = nil,
    resultHandler: @escaping (_ result: T?, _ error: Error?) -> Void) -> URLSessionTask? { ... }

public func invoke(_ method: Method, path: String,
    arguments: [String: Any] = [:], content: Data? = nil, contentType: String? = nil,
    responseHandler: @escaping (_ content: Data, _ contentType: String?) throws -> T?,
    resultHandler: @escaping (_ result: T?, _ error: Error?) -> Void) -> URLSessionTask? { ... }

All three methods accept the following arguments:

  • method – the HTTP method to execute
  • path – the path to the requested resource
  • arguments – a dictionary containing the method arguments as key/value pairs
  • content – an optional Data instance representing the body of the request
  • contentType – an optional string value containing the MIME type of the content
  • resultHandler – a callback that will be invoked upon completion of the method

The first version of the method uses JSONSerialization to decode response data. The second uses JSONDecoder to return a decodable value. The third version accepts an additional responseHandler argument to facilitate decoding of custom response content (for example, a UIImage).

All three methods return an instance of URLSessionTask representing the invocation request. This allows an application to cancel a task, if necessary.

Arguments

Like HTML forms, arguments are submitted either via the query string or in the request body. Arguments for GET, PUT, PATCH, and DELETE requests are always sent in the query string.

POST arguments are typically sent in the request body, and may be submitted as either “application/x-www-form-urlencoded” or “multipart/form-data” (determined via the service proxy’s encoding property). However, if a custom body is specified via the content parameter, POST arguments will be sent in the query string.

Any value that provides a description property may be used as an argument. This property is generally used to convert the argument to its string representation. However, Date instances are automatically converted to a 64-bit integer value representing epoch time (the number of milliseconds that have elapsed since midnight on January 1, 1970).

Additionally, array instances represent multi-value parameters and behave similarly to tags in HTML. Further, when using the multi-part form data encoding, instances of URL represent file uploads and behave similarly to tags in HTML forms. Arrays of URL values operate similarly to tags.

Return Values

The result handler is called upon completion of the operation. If successful, the first argument will contain a deserialized representation of the content returned by the server, and the second argument will be nil. Otherwise, the first argument will be nil, and the second will be populated with an Error instance describing the problem that occurred.

Note that, while service requests are typically processed on a background thread, result handlers are always executed on the application’s main thread. This allows result handlers to update the user interface directly, rather than posting a separate update operation to the main queue.

If the server returns an error response, a localized description of the error will be provided in the localized description of the error parameter. Further, if the error is returned with a content type of “text/plain”, the response body will be returned in the error’s debug description.

Example

The following code snippet demonstrates how the WebServiceProxy class might be used to access the operations of a simple math service:

// Create service proxy
let webServiceProxy = WebServiceProxy(session: URLSession.shared, serverURL: URL(string: "http://localhost:8080")!)

// Get sum of "a" and "b"
webServiceProxy.invoke(.get, path: "/math/sum", arguments: [
    "a": 2,
    "b": 4
]) { (result: Int?, error: Error?) in
    // result is 6
}

// Get sum of all values
webServiceProxy.invoke(.get, path: "/math/sum", arguments: [
    "values": [1, 2, 3, 4]
]) { (result: Int?, error: Error?) in
    // result is 10
}

Additional Information

This article introduced the Kilo framework and provided an overview of its key features. For additional information, see the the project README.

 

Dynamically Loading Table View Images in iOS

NOTE 7/2/2018 The WSWebServiceProxy class mentioned in this article is now part of the Kilo project.

iOS applications often display thumbnail images in table views alongside other text-based content such as contact names or product descriptions. However, these images are not usually delivered with the initial request, but must instead be retrieved separately afterward. They are typically downloaded in the background as needed to avoid blocking the main thread, which would temporarily render the user interface unresponsive.

For example, consider this REST service, which returns a list of simulated photo data:

[
  {
    "albumId": 1,
    "id": 1,
    "title": "accusamus beatae ad facilis cum similique qui sunt",
    "url": "http://placehold.it/600/92c952",
    "thumbnailUrl": "http://placehold.it/150/92c952"
  },
  {
    "albumId": 1,
    "id": 2,
    "title": "reprehenderit est deserunt velit ipsam",
    "url": "http://placehold.it/600/771796",
    "thumbnailUrl": "http://placehold.it/150/771796"
  },
  {
    "albumId": 1,
    "id": 3,
    "title": "officia porro iure quia iusto qui ipsa ut modi",
    "url": "http://placehold.it/600/24f355",
    "thumbnailUrl": "http://placehold.it/150/24f355"
  },
  ...
]

Each record contains a photo ID, album ID, and title, as well as URLs for both thumbnail and full-size images; for example:

Photo Class

A class representing this data might be defined as follows:

class Photo: NSObject {
    var id: Int = 0
    var albumId: Int = 0
    var title: String?
    var url: URL?
    var thumbnailUrl: URL?

    init(dictionary: [String: Any]) {
        super.init()

        setValuesForKeys(dictionary)
    }

    override func setValue(_ value: Any?, forKey key: String) {
        switch key {
        case #keyPath(url):
            url = URL(string: value as! String)

        case #keyPath(thumbnailUrl):
            thumbnailUrl = URL(string: value as! String)

        default:
            super.setValue(value, forKey: key)
        }
    }
}

Since instances of this class will be populated using dictionary values returned by the web service, an initializer that takes a dictionary argument is provided. The setValuesForKeys(_:) method of NSObject is used to map dictionary entries to property values. The ID and title properties are handled automatically by this method; the class overrides setValue(_:forKey:) to convert the URLs from strings into actual URL instances.

View Controller Class

A basic user interface for displaying service results in a table view is shown below:

Row data is stored in an array of Photo instances. Previously loaded thumbnail images are stored in a dictionary that associates UIImage instances with photo IDs:

class ViewController: UITableViewController {
    // Row data
    var photos: [Photo]!

    // Image cache
    var thumbnailImages: [Int: UIImage] = [:]

    ...    
}

The photo list is loaded the first time the view appears. The WSWebServiceProxy class of the open-source HTTP-RPC library is used to retrieve the data. If the call succeeds, the response data (an array of dictionary objects) is transformed into an array of Photo objects and the table view is refreshed. Otherwise, an error message is displayed:

override func viewWillAppear(_ animated: Bool) {
    super.viewWillAppear(animated)

    // Load photo data
    if (photos == nil) {
        let serviceProxy = WSWebServiceProxy(session: URLSession.shared, serverURL: URL(string: "https://jsonplaceholder.typicode.com")!)

        serviceProxy.invoke("GET", path: "/photos") { result, error in
            if (error == nil) {
                let photos = result as! [[String: Any]]

                self.photos = photos.map({
                    return Photo(dictionary: $0)
                })

                self.tableView.reloadData()
            } else {
                let alertController = UIAlertController(title: "Error", message: error!.localizedDescription, preferredStyle: .alert)

                alertController.addAction(UIAlertAction(title: "OK", style: .default))

                self.present(alertController, animated: true)
            }
        }
    }
}

Cell content is generated as follows. The corresponding Photo instance is retrieved from the photos array and used to configure the cell:

override func tableView(_ tableView: UITableView, numberOfRowsInSection section: Int) -&gt; Int {
    return (photos == nil) ? 0 : photos.count
}

override func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -&gt; UITableViewCell {
    var cell = tableView.dequeueReusableCell(withIdentifier: cellIdentifier)

    if (cell == nil) {
        cell = UITableViewCell(style: .default, reuseIdentifier: cellIdentifier)
    }

    let photo = photos[indexPath.row];

    cell!.textLabel!.text = photo.title

    // Attempt to load image from cache
    cell!.imageView!.image = thumbnailImages[photo.id]

    if (cell!.imageView!.image == nil &amp;&amp; photo.thumbnailUrl != nil) {
        // Image was not found in cache; load it from the server

        ...
    }

    return cell!
}

If the thumbnail image is already available in the cache, it is used to populate the cell’s image view. Otherwise, it is loaded from the server and added to the cache as shown below. If the cell is still visible when the image request returns, it is updated immediately and reloaded:

let serverURL = URL(string: String(format: "%@://%@", photo.thumbnailUrl!.scheme!, photo.thumbnailUrl!.host!))!

let serviceProxy = WSWebServiceProxy(session: URLSession.shared, serverURL: serverURL)

serviceProxy.invoke("GET", path: photo.thumbnailUrl!.path) { result, error in
    if (error == nil) {
        // If cell is still visible, update image and reload row
        let cell = tableView.cellForRow(at: indexPath)

        if (cell != nil) {
            if let thumbnailImage = result as? UIImage {
                self.thumbnailImages[photo.id] = thumbnailImage

                cell!.imageView!.image = thumbnailImage

                tableView.reloadRows(at: [indexPath], with: .none)
            }
        }
    } else {
        print(error!.localizedDescription)
    }
}

Finally, if the system is running low on memory, the image cache is cleared:

override func didReceiveMemoryWarning() {
    super.didReceiveMemoryWarning()

    // Clear image cache
    thumbnailImages.removeAll()
}

Summary

This article provided an overview of how images can be dynamically loaded as needed to populate table view cells in iOS. For more ways to simplify iOS app development, please see my projects on GitHub:

  • MarkupKit – declarative UI for iOS and tvOS
  • Kilo – Lightweight REST for iOS and tvOS