Pagination on arrays – MongoDB

In this tutorial, we will teach you, how you can do pagination on arrays of documents in MongoDB.

If you have a very large database, fetching all the documents in one query might be very slow. But we have a solution: Pagination. It allows you to fetch a few records in one query and the next few in another query. For example, if you have 1000 users, the first query will fetch users from 0 to 100, the second query will fetch users from 101 to 200, and so on.

Problem

Pagination comes with a problem; when data is stored in arrays. In Mongo DB, the data is stored in JSON format. You might be saving data in arrays. For example, take the following document:

db.collection("users").insertOne({
	name: "Adnan",
	workouts: [{
		name: "Push ups",
		sets: 10,
		reps: 30
	}, {
		name: "Pull ups",
		sets: 7,
		reps: 10
	}, {
		name: "Crunches",
		sets: 5,
		reps: 50
	}, {
		name: "Deadlifts",
		sets: 3,
		reps: 15
	}, {
		name: "Shoulder press",
		sets: 8,
		reps: 8
	}]
})

Here, one document has an array of “workouts” and it might have a lot more data in it. For example, the user we created above has 5 “workouts” array elements, but you want to show 2 workouts on each query. If you run the regular query to fetch the users document, then it will return all the 5 workouts:

// not a solution

// return the whole object
const data = await db.collection("users").find({
	name: "Adnan"
}).toArray()
console.log(data[0])

But it will return the whole object with all 5 “workouts” elements, which is not the desired outcome.

So how will you paginate on that ?

Solution

This is where the Mongo DB $slice operator comes in. It works the same as the Javascript slice() function. It cuts the array from provided start and end values. So we will use the following query to get the users document but with a few workouts:

// solution

// return the first 2 workouts
const data = await db.collection("users").find({
	name: "Adnan"
}, {
	projection: {
		workouts: {
			$slice: [0, 2]
		}
	}
}).toArray()
console.log(data[0])

Slice start value works on indexes, so 0 means the first element of the array, and the end means the number of elements to fetch. So it will fetch 2 array elements starting at index 0.

The second parameter to the find() function tells the projection, which means which keys you want to fetch. You have to write all your key names inside the “projection” object. Now if you want to fetch the next 2 records, you can simply do the following:

// return the workouts from 2 to 4 
const data = await db.collection("users").find({
	name: "Adnan"
}, {
	projection: {
		workouts: {
			$slice: [2, 2]
		}
	}
}).toArray()
console.log(data[0])

The “start” is set to 2, which means the array starts from index 2 which will be the 3rd element of the array. Because the first and second are already fetched in the previous query. The “end” is again set to 2 so we are again fetching the 2 records from the array.

That’s how you can keep going on and implementing pagination on your document’s arrays in MongoDB. If you face any problem in following this, kindly do let me know.

[wpdm_package id=’2045′]