In Early September 2020 Google released the first Alpha of a new Jetpack library named DataStore. DataStore is a replacement for SharedPreferences. While there are some similarities to SharedPreferences, DataStore offers far greater flexibility. In this series of posts we’ll take a detailed look at this new library.
NOTE: There is an update to this series of articles that covers how to use newer releases of DataStore.
We’ve looked at various aspects of DataStore during this series, but we haven’t looked at the benefits of Protobuf. The old SharedPreferences is storage of key / value pairs. But Protobuf allows us to persist complex, structured, type-safe data. In this article we’ll look at some of the possibilities that this opens up.
Before we continue, it is worth mentioning that this focuses on the proto3 syntax for Protobuf models. It is perfectly acceptable to use proto2 also – that has not been deprecated. Also, I am using Wire for the codegen, and the behaviour of the models created may differ from other tools.
Message
A Kotlin model created from a Protobuf message by Wire is similar to a Kotlin data class. It isn’t actually implemented as a data class, but the behaviour is very similar. It consists of a number of immutable fields, and may be primitive types, other proto types, and collections.
Each field in a message has a unique field number, and is defined thus:
= ;
So if we look back at SimpleData that we have used up to now we can see a single string field named text which has a field number of 1:
syntax = "proto3";
option java_package = "com.stylingandroid.datastore.data";
option java_multiple_files = true;
message SimpleData {
string text = 1;
}
These field numbers are written along with the value. This protects the integrity of the data structure, irrespective of the order in which fields are persisted. This means that we can have different Protobuf implementations writing the data, and then reading it back again.
Field Types
Protobuf models have some fundamental types known as scalar type. These are the equivalent of the primitive types that as have in Java / Kotlin. The proto3 documentation contains a full list of these along with their equivalents in various languages (Java included).
As well as using scalar types for fields, we can also use other proto message types:
syntax = "proto3";
option java_package = "com.stylingandroid.datastore.data";
import "SimpleData.proto";
message ComplexData {
string text = 1;
SimpleData simpleData = 2;
.
.
.
}
Here we are using SimpleData as a field type. The only thing that we need to bear in mind is to include the proto definition to be able to use it.
Enum
Another useful type that we can use for field types is an enum. This is very similar to a Kotlin Enum, even though it isn’t implemented as one by Wire. We can define these as either internally within our message or externally:
syntax = "proto3";
option java_package = "com.stylingandroid.datastore.data";
import "SimpleData.proto";
message ComplexData {
string text = 1;
SimpleData simpleData = 2;
enum Internal {
ZERO = 0;
ONE = 1;
TWO = 2;
}
Internal internal = 3;
External external = 4;
.
.
.
}
enum External {
ZERO = 0;
ONE = 1;
TWO = 2;
}
We can also define an external enum in a separate proto file, but we must include it in order to use it.
Irrespective of whether we define an enum internally or externally, the type declaration and the field declaration are separate. We declare the Internal type within ComplexData, but we only get a field of type Internal because of the field declaration on line 14.
Collections
There are a couple of simple collection types available to us – a list and a map:
syntax = "proto3";
option java_package = "com.stylingandroid.datastore.data";
import "SimpleData.proto";
message ComplexData {
string text = 1;
SimpleData simpleData = 2;
enum Internal {
ZERO = 0;
ONE = 1;
TWO = 2;
}
Internal internal = 3;
External external = 4;
repeated string names = 5;
map ages = 6;
.
.
.
}
enum External {
ZERO = 0;
ONE = 1;
TWO = 2;
}
The repeated qualifier defines a simple list. In this case, because we have specified a type of string the generated Kotlin field will be of type List<String>.
The map type should be easy enough to understand as the Kotlin field that Wire generates is of type Map<String, Int>.
While both of these appear pretty straightforward, having them available offers us possibilities that were never easy using key / value based SharedPreferences. To achieve that kind of behaviour we always needed to use potentially error-prone techniques such as flattening a collection to a string .
Oneof
oneof fields initially seem a little odd because there is no direct equivalent in Kotlin. They are closest to a union in C or C++, but even that is not identical to how they work in proto models. Essentially a oneof block enables us to specify a number of fields, and only one of them may hold a value at any given time. An example will help explain this better:
syntax = "proto3";
option java_package = "com.stylingandroid.datastore.data";
import "SimpleData.proto";
message ComplexData {
string text = 1;
SimpleData simpleData = 2;
enum Internal {
ZERO = 0;
ONE = 1;
TWO = 2;
}
Internal internal = 3;
External external = 4;
repeated string names = 5;
map ages = 6;
oneof myOneof {
string oneofString = 7;
int32 oneofInt = 8;
}
}
enum External {
ZERO = 0;
ONE = 1;
TWO = 2;
}
Here we have a oneof block consisting of two fields: oneofString and oneofInt. Initially this appears to be a nested object, but these fields are actually direct children of ComplexData, and thus must have unique field numbers within ComplexData.
The Kotlin code that Wire generates has both of these fields as nullable. So oneofString is declared as a String?; and oneofInt is declared as an Int?. The protobuf documentation states:
Setting a oneof field will automatically clear all other members of the oneof
The Kotlin code that Wire generated doesn’t quite behave like this, although the fundamental rule is still true. The generated ComplexData Kotlin class defines all of the fields as val – meaning that they are immutable. Therefore we cannot actually call a setter on any of the oneof fields. To modify the ComplexData we have to create a copy overriding the values for specific fields:
val newComplexData = complexData.copy(
oneofString = "The meaning of life, the universe, and everything"
)
Where the Wire Kotin models enforce the oneof behaviour is if we attempt to give more than one field within a given oneof block values:
// WARNING: This will throw an exception
val newComplexData = complexData.copy(
oneofString = "The meaning of life, the universe, and everything",
oneofInt = 42
)
As the comment suggest this results in an exception:
java.lang.IllegalArgumentException: At most one of oneofString, oneofInt may be non-nullSo the behaviour dictated by the oneof block gets enforced during the creation of a new instance of ComplexData rather than via setter behaviour.
Conclusion
This is a quick summary of some of the features of Protobuf models, and how they allow us to structure data. The official language guide gives far more detail.
However, it should be fairly clear how structured, complex models enable us to now persist data in a more meaningful way rather than having to coerce them in to the subset of primitive types supported by SharedPreferences. We can now define a data type which models the user-preferences model required by our app. If we model this using proto3 we obtain far better structured data when we read from DataStore compared to SharedPreferences.
That concludes out look at what DataStore can offer us. It enables us to have far more structured data, around which we can easily add an encryption wrapper to protect the security of the data being persisted. We can also chose between different codegen tools to enable us to chose whichever best suits our needs.
The source code for this article is available here.
© 2020 – 2021, Mark Allison. All rights reserved.
Copyright © 2020 Styling Android. All Rights Reserved.
Information about how to reuse or republish this work may be available at http://blog.stylingandroid.com/license-information.
