Working around the AWS AppSync fetch limits in your iOS app
Earlier this year, I decided to migrate from a self-hosted backend to AWS for my app, Plate-It. It was a no-brainer for me: the performance improvements, cost savings, one less codebase to maintain, and the opportunity to learn about a new cloud platform were just some of the benefits I identified. The transition was seamless, however I had to become accustomed to some of the limitations of using the cloud, which I no longer had full control over.
One such limitation was the fetch limit when making use of the
AWSAppSync
library. By default the limit is set to 20, but you can override this
when you initialise your ListQuery object. What isn’t clear is that
you can’t fetch any more than 1000 items at once, as specified in the
AWS AppSync
‘Iterations in a foreach loop in mapping templates’
quota.
What’s even more frustrating, is that the limitation is imposed before your records are filtered. For example, lets say that you are querying for records with an owner of ‘joebloggs’ and you have set the limit to 100. The query pulls the first 100 records from DynamoDB, but only 4 of the records have an owner of ‘joebloggs’. After the filter expression ‘owner = joebloggs’ is applied you will only be given those 4 records, even though you might expect to receive 100 records.
When your query is returned you’ll also receive a nextToken
. This is
used for if you want to query the next 100 records, and so on until
there are no more records to query. In a UIKit app, you might have a
UITableView
that has a UITableViewCell
for each record that is
returned in each query. When a user scrolls to the bottom of that
UITableView
, you would use that nextToken
to query the next 100
records. From my personal experience, users didn’t want to keep
scrolling to the bottom in order to see more of their data, they just
wanted it to all load at once.
How to work around the limits#
In this example, I have a DynamoDB table called Order
, which
contains many Order
records for an online store. In the iOS app a
user can view all of the orders they have placed in the past. Assuming
you have already set up your AWSAppSyncClient
in your AppDelegate
,
here’s how you can go about fetching all your data at once:
1. Set up your class#
Here I’m creating a singleton, although you should ensure you research
the pros and cons of using this approach. My class is called
OrderService
, and I can put all of the CRUD (Create, Read, Update,
Delete) methods that interact with the Order
AppSync API in here.
I also create an orders
array for storing the data once it has been
fetched.
class OrderService {
static let shared = OrderService()
private var appSyncClient: AWSAppSyncClient?
public var orders: [Order] = []
private init() {
let appDelegate = UIApplication.shared.delegate as! AppDelegate
appSyncClient = appDelegate.appSyncClient
}
}
2. Create your fetchOrders() method#
My method is going to work by recursively calling itself, passing in
the nextToken
as an argument. The nextToken
argument is optional,
which is okay because the ListQuery
initialiser also takes in an
optional nextToken
argument. When making your first call to
fetchOrders
, if no nextToken
argument is provided, we can assume
that the user wants to re-fetch all data from the server again.
Therefore, if nextToken
is nil, we will empty our orders
array to
ensure we don’t accidentally show duplicate results.
As we want to fetch all of our user’s data with as few queries as
possible, I have set the ListOrdersQuery
limit to 1000 (which is the
maximum). Setting the limit to a number higher than 1000 won’t result
in an error, but will simply query using the default limit (20).
Once we have cycled through our fetched items, initialised their
respective objects, and added them to the orders
array, we check to
see if we were given a nextToken
indicating that there are more
records to query. If so, we will recursively call our fetchOrders
method, passing in the nextToken
as an argument. However, if no
nextToken
is provided, we know that we have queried every record in
the table. We therefore call the completion handler, sending a signal
to our View Controller (or any other caller) that we have finished
fetching all of our Orders. In this case, we use the Swift-provided
Result
enum, which is handy for indicating a failure (if an error
exists) or a success (once all of our orders have been fetched).
func fetchOrders(
nextToken: String?=nil,
_ completion: @escaping (Swift.Result<String, Error>) -> Void
) {
if nextToken == nil { orders = [] }
let query = ListOrdersQuery(limit: 1000, nextToken: nextToken)
appSyncClient?.fetch(query: query, cachePolicy: .fetchIgnoringCacheData) { result, error in
if let error = error {
completion(.failure(error))
} else {
if let items = result?.data?.listOrders?.items {
for item in items {
self.orders.append(Order(
id: item?.id,
sku: item?.sku,
price: item?.price
))
}
}
if let nextToken = result?.data?.listOrders?.nextToken {
self.fetchOrders(nextToken: nextToken) { result in
completion(result)
}
} else {
completion(.success("Completed fetching!")) }
}
}
}
Conclusion#
That concludes this tutorial. If you have any questions, feel free to reach out to me via Twitter, and keep an eye open for my upcoming tutorials.
The completed solution can be found here on GitHub.