Muselee 11: Repository – Part 1

Muselee is a demo app which allows the user to browse popular music artists. It is not intended to be a fully-featured user app, but a vehicle to explore good app architecture, how to implement current best-practice, and explore how the two often go hand in hand. Moreover it will be used to explore how implementing some specific patterns can help to keep our app both maintainable, and easy to extend.

One issue that we have with Muselee so far is that it is not very efficient in how it retrieves data. Each time the user opens the app, we retrieve the latest data from the last.fm top artists API. While this is functional, there are a few reasons why this isn’t ideal behaviour.

Firstly, it does not work well if the user moves in to an area with poor network connectivity – it may not be possible to retrieve the data, and we therefore cannot show anything. While this will not cause the app to crash, it’s not a great user experience.

Secondly, some third-party APIs may put restrictions on how often we may retrieve data, and may return errors if we exceed that. If this happens then, once again, the app will not crash, but the user experience will be poor.

Thirdly, although the last.fm data does update quite regularly, it is unlikely to change on a minute-by-minute basis unlike, for example, a sports match score, or live public transport arrival departure data. However, we may be requesting the data every few minutes if the user opens the app frequently, and to fetch fresh data which is unlikely to have changed is both wasteful of the user’s data, if they are on a metered network; and will cause them to wait while we retrieve that data. Again this is a poor user experience.

We can address this by implementing a simple repository pattern which will store the data on the user’s device, and used this cached version if the user returns to the app before this cached data expires. This will limit how often we actually request fresh data from the backend and so will help us to keep within the API usage limits, limit how much of the user’s data we require, and improve wait times. It will even allow for lack of connectivity, but only when the cached data is within its validity period. However, that is still an improvement on the previous no-connectivity behaviour. Of course, we could get a bit cleverer in our approach to this to re-use expired data if we have it, but we’ll keep it simple to start with.

Many third-party APIs will include meta data which indicates how long the data should be considered valid. Typically this will be using the HTTP Expires header in the response. However we could also use the Cache-Control header to determine the maximum amount of time a cache should serve up a cached version of the data. In our case the Repository is essentially a local cache so we, can fully leverage this for the purpose for which it is intended.

The response we get from last.fm doesn’t contain either Expires or Cache-Control headers which might indicate how long we should consider the content valid for, it does return the following response header:

Access-Control-Max-Age: 86400

Technically this header isn’t applicable to Muselee because it is not using CORS, but we can use this header to get a hint of how long we can consider the response valid for. This value is in seconds, and we will use this to calculate the validity for the content. However it is good practise to prefer the other headers if they exist so we can create a fallback to determine the validity period:

class LastFmTopArtistsProvider(
    private val topArtistsApi: LastFmTopArtistsApi,
    private val connectivityChecker: ConnectivityChecker,
    private val mapper: DataMapper, List>
) : DataProvider {

    override fun requestData(callback: (topArtists: TopArtistsState) -> Unit) {
        if (!connectivityChecker.isConnected) {
            callback(TopArtistsState.Error("No network connectivity"))
            return
        }
        callback(TopArtistsState.Loading)
        topArtistsApi.getTopArtists().enqueue(object : Callback {
            override fun onFailure(call: Call, t: Throwable) {
                callback(TopArtistsState.Error(t.localizedMessage))
            }

            override fun onResponse(call: Call, response: Response) {
                response.body()?.also { topArtists ->
                    callback(TopArtistsState.Success(mapper.encode(topArtists to response.expiry)))
                }
            }
        })
    }

    private val Response.expiry: Long
        get() {
            val expires: Long? = if (headers().names().contains(HEADER_EXPIRES)) {
                HttpDate.parse(headers().get(HEADER_EXPIRES)).time
            } else null
            val cacheControlMaxAge = raw().cacheControl().maxAgeSeconds().toLong()
            val maxAge: Long? =
                cacheControlMaxAge.takeIf { it >= 0 } ?: headers().get(HEADER_AC_MAX_AGE)?.toLong()
            val date = if (headers().names().contains(HEADER_DATE)) {
                HttpDate.parse(headers().get(HEADER_DATE)).time
            } else {
                System.currentTimeMillis()
            }
            return expires
                ?: maxAge?.let { date + TimeUnit.SECONDS.toMillis(it) }
                ?: date + TimeUnit.DAYS.toMillis(1)
        }

    companion object {
        private const val HEADER_DATE = "Date"
        private const val HEADER_EXPIRES = "Expires"
        private const val HEADER_AC_MAX_AGE = "Access-Control-Max-Age"
    }
}

This will use the Expires header if it exists, otherwise it will use the Cache-Control header if that exists, otherwise it will use the Access-Control-Max-Age if that exists, otherwise it will default to one day.

This expiry time is then passed to the mapper which converts this to the domain model:

class LastFmArtistsMapper : DataMapper, List> {

    override fun encode(source: Pair): List {
        val (lastFmArtists, expiry) = source
        return lastFmArtists.artists.artist.map { artist ->
            Artist(artist.name, artist.normalisedImages(), expiry)
        }
    }

    private fun LastFmArtist.normalisedImages() =
        images.map { it.size.toImageSize() to it.url }.toMap()

    private fun String.toImageSize(): ImageSize =
        when (this) {
            "small" -> ImageSize.SMALL
            "medium" -> ImageSize.MEDIUM
            "large" -> ImageSize.LARGE
            "extralarge" -> ImageSize.EXTRA_LARGE
            else -> ImageSize.UNKNOWN
        }
}

We also update the Artist data class to take an expiry:

data class Artist(val name: String, val images: Map, val expiry: Long) {

    enum class ImageSize {
        SMALL,
        MEDIUM,
        LARGE,
        EXTRA_LARGE,
        UNKNOWN
    }
}

So now our domain `Artist` object contains information about how long it should be considered valid, and the rest of the module is completely agnostic of how that expiry was determined.

The next thing that we need to do is to define how we will store the data. I have opted to use Room, and this implementation will essentially be in the outer layout of our clean architecture because it is interfacing with external components – in this case an Android SQLite database, albeit through Room. First we need to define our data model for how we will persist the data:

@Entity(indices = [Index(value = ["name"])])
data class DbArtist(
    @PrimaryKey val rank: Int,
    val name: String,
    val created: Long,
    val expiry: Long
)

@Entity(
    primaryKeys = ["rank", "typeIndex"],
    foreignKeys = [
        ForeignKey(
            entity = DbArtist::class,
            parentColumns = ["rank"],
            childColumns = ["rank"],
            onDelete = ForeignKey.CASCADE
        )
    ]
)
data class DbImage(
    val rank: Int,
    val typeIndex: Int,
    val url: String
)

Here we are creating two tables, one to hold the artist information, and a second to hold the image information. A single Artist can have multiple different sized images, and this allows us to create that one-to-many relationship. We have a primary key which is based upon the rank of the artist (i.e. the most popular having a rank of 1), and index the images to the rank of the associated artist, with a compound primary key because each artist will have one image of each type, therefore the combination of rank and typeIndex will be unique. We also define a foreign key which will cause the associated images to be deleted if a specific artist is deleted.

Next we create our DAO which defines the database operations that we can perform:

@Dao
interface TopArtistsDao {

    @Insert
    fun insertTopArtists(artists: List)

    @Insert
    fun insertImages(artists: List)

    @Query("SELECT * FROM DbArtist")
    fun getAllArtists(): List

    @Query("SELECT * FROM DbImage")
    fun getAllImages(): List

    @Query("DELETE FROM DbArtist WHERE expiry < :target")
    fun deleteOutdated(target: Long)

    @Query("DELETE From DbArtist")
    fun deleteAll()
}

So we have operations to add and retrieve lists of both DbArtist and DbImage records, plus a couple of delete operations. deleteOutdated() is the only one worthy of much explanation because it is what will be used to delete any expired content. We pass in a timestamp as an argument, and it will delete any artists whose expiry is earlier than the timestamp.

Next we define our Database which associates the data model with the DAO and associates it with the SQLite database:

@Database(entities = [DbArtist::class, DbImage::class], version = 1, exportSchema = false)
abstract class TopArtistsDatabase : RoomDatabase() {

    abstract fun topArtistsDao(): TopArtistsDao
}

Next we need to revise the DataMapper interface that we defined in the core module, because we now need bi-directional mapping:

interface DataMapper {
    fun encode(source: S): R
    fun decode(source: R): S = throw NotImplementedError()
}

We can now implement a bi-directional mapper to convert between the database data model and the domain data model:

class DatabaseTopArtistsMapper : DataMapper, Pair>> {
 
    override fun encode(source: Triple): Pair> {
        val (rank, artist, created) = source
        return DbArtist(
            rank,
            artist.name,
            created,
            artist.expiry
        ) to artist.images.map { DbImage(rank, it.key.ordinal, it.value) }
    }
 
 
    override fun decode(source: Pair>): Triple {
        val (artist, images) = source
        return Triple(
            artist.rank,
            Artist(
                artist.name,
                images.map { Artist.ImageSize.values()[it.typeIndex] to it.url }.toMap(),
                artist.expiry
            ),
            artist.created
        )
    }
}

This will convert between an Artist object, which contains a collection of Image objects, to a DbArtist and a collection of associated Images.

To tie all of this together, we are going to create a new interface in the <code>core</code> module named DataPersister:

interface DataPersister : DataProvider {
    fun persistData(data: T)
}

This extends the existing DataProvider and adds a method to persist data as well as retrieving it.

We can now tie all of this together in our implementation of DataPersister:

class DatabaseTopArtistsPersister(
    private val dao: TopArtistsDao,
    private val mapper: DataMapper, Pair>>
) : DataPersister> {

    override fun persistData(data: List) {
        dao.deleteAll()
        val now = System.currentTimeMillis()
        val dbData = data.mapIndexed { index, artist ->
            mapper.encode(Triple(index, artist, now))
        }
        dao.insertTopArtists(dbData.map { it.first })
        dao.insertImages(dbData.flatMap { it.second })
    }

    override fun requestData(callback: (item: List) -> Unit) {
        dao.deleteOutdated(System.currentTimeMillis())
        val dbImages = dao.getAllImages()
        val artists = dao.getAllArtists().sortedBy { it.rank }.map { artist ->
            mapper.decode(artist to dbImages.filter { it.rank == artist.rank }).second
        }
        callback(artists)
    }
}

persistData() will delete any existing data and convert the supplied domain objects to database objects and save them to the database. requestData() will first delete any expired records, and then retrieve the data from the database, and convert the database objects to domain objects.

The interesting thing here is how simple this makes our logic. The Persister is responsible for deleting expired objects, but not for determining how long the expiry period should be - that is the responsibility of the network components based on the server response. But this demonstrates how putting the responsibility in the correct place, and passing the pertinent information through the system, in this case through an expiry field, can make it possible to put the responsibilities for determining validity and purging expired data in to separate components whilst keeping them completely agnostic of one another.

While implementing the repository itself seems fairly trivial now that we have the outer layer components in place and ready to go, there is a slight complication and we'll explore this in the next article.

Although this isn't yet hooked up and working, the source code for this article is available here.

© 2019, Mark Allison. All rights reserved.

Copyright © 2019 Styling Android. All Rights Reserved.
Information about how to reuse or republish this work may be available at http://blog.stylingandroid.com/license-information.