Soft Delete 🗑 – Node.js, MongoDB

In this article, we will teach you, how you can implement soft delete in your Node.js app with MongoDB as database.

We also created a tutorial on how to do soft delete in Laravel, check our tutorial here.

Soft delete 🗑 is useful when you do not want to actually delete the record from database, but mark the data as deleted.

You might have seen “This message has been deleted” on WhatsApp đŸ’Ŧ when you delete a message. That was soft delete. Because if you actually delete the message from database, then you would have no reference if there was a message at that time.

It is also helpful in cases where you want to allow the user to delete some data, but wants the administrator 👨đŸģ‍đŸ’ŧ to see the data. So when user deletes the data, it will not actually delete the data from database. It will simply hide it from user, but administrator can still see it. Only administrator can actually delete the data.

We will use “users” as entity for the sake of this tutorial.

In this article, you will learn:

  1. Insert users in MongoDB
  2. Fetch non-trashed users
  3. Soft delete a user
  4. Fetch trashed users
  5. Restore a user
  6. Permanently delete a user

1. Insert users in MongoDB ➕

Following API will insert a new document in MongoDB “users” collection using Node.js:

// API to insert data
app.post("/insert", async function (request, result) {
    // Get input fields
    const name = request.fields.name || "";

    // Insert data in database
    await db.collection("users")
        .insertOne({
            name: name,
            deletedAt: null // This will save the time when user was deleted
        });

    // Return response to the client
    result.json({
        status: "success",
        message: "User has been inserted."
    });
});

It will get user name from client app and insert a new document in “users” collection. Note that it creates another field “deletedAt” and set it’s value to null. Having NULL value means that the user is not deleted.

2. Fetch non-trashed users

Then we are going to fetch all the users that are not deleted.

// API to fetch non-trashed users
app.post("/fetch", async function (request, result) {
    const users = await db.collection("users")
        .find({
            deletedAt: null
        }).toArray();

    result.json({
        status: "success",
        message: "Users has been fetched.",
        users: users
    });
});

Here we are using filter to fetch all the records where deletedAt field is null.

3. Soft delete a user ␥

Now we will create an API that will soft delete a user.

// API to move the user to trash can
app.post("/delete", async function (request, result) {
    // Get user ID
    const _id = request.fields._id || "";

    // Don't delete the document,
    // just update the deletedAt field
    await db.collection("users")
        .updateOne({
            _id: ObjectId.createFromHexString(_id)
        }, {
            $set: {
                deletedAt: new Date()
            }
        });

    result.json({
        status: "success",
        message: "User has been moved to trash can."
    });
});

Here, we are first getting the user ID from client app. Then we are setting the deletedAt field to the current date.

If you run the /fetch API again, you will not see that user’s data in the response. Even though the record exists in the database, it will not be returned.

4. Fetch trashed users

Now that we have trashed the user, we need to find a way to see all those trashed users. So we will create an API that will return all the users that has been soft-deleted.

// API to see all trashed users
app.post("/trashed", async function (request, result) {
    const users = await db.collection("users")
        .find({
            deletedAt: {
                $ne: null
            }
        }).toArray();

    result.json({
        status: "success",
        message: "Trashed users has been fetched.",
        users: users
    });
});

At this point, you can give administrator 2 options, either:

  1. Restore the user
  2. Delete the user permanently

5. Restore a user â†Šī¸

Restoring the user will simply set the deletedAt field of user back to null, like it was when user was created.

// API to restore a user
app.post("/restore", async function (request, result) {
    const _id = request.fields._id || "";

    await db.collection("users")
        .updateOne({
            _id: ObjectId.createFromHexString(_id)
        }, {
            $set: {
                deletedAt: null
            }
        });

    result.json({
        status: "success",
        message: "User has been restored."
    });
});

This will also receive a user ID that needs to be restored. If you call the /fetch API again, you will see that user in the response.

6. Permanently delete a user 🗑

Permanently deleting a user will actually deletes the user from database. There will no way to restore it once done (unless you have a backup 🎒 of your database).

// API to actually delete the user from database
app.post("/delete-permanently", async function (request, result) {
    const _id = request.fields._id || "";

    await db.collection("users")
        .deleteOne({
            $and: [{
                _id: ObjectId.createFromHexString(_id)
            }, {
                deletedAt: {
                    $ne: null
                }
            }]
        });

    result.json({
        status: "success",
        message: "User has been permanently deleted."
    });
});

Usually, this access is given to super admin only.

Bonus â­ī¸

One more thing you can do, is if you have saved the user ID in other collections as well, then you can save the data in the dependent collections before removing them.

For example, if you have an “orders” collection, where you are saving user’s orders in the following format:

{
    "_id": ObjectId("1234567890"),
    "userId": ObjectId("0987654321")
}

If you permanently delete the user, then this order’s data will be lost as well. So best practice is to:

  1. Fetch the user.
  2. Put in orders collection.
  3. Delete the user.

Following is the code for that:

const user = await db.collection("users")
    .findOne({
        _id: ObjectId.createFromHexString(_id)
    });

if (user == null) {
    result.json({
        status: "error",
        message: "User not found."
    });
    return;
}

await db.collection("orders")
    .updateMany({
        userId: user._id
    }, {
        $set: {
            user: {
                _id: user._id,
                name: user.name
            }
        }
    });

Now, when you delete the user, your “orders” collection will save the user data for historical use.

{
    "_id": ObjectId("1234567890"),
    "userId": ObjectId("0987654321"),
    "user": {
        "_id": ObjectId("0987654321"),
        "name": "Adnan"
    }
}

Conclusion 📝

So that’s it. That’s how you can create a soft delete feature in your Node.js application with MongoDB as backend. It is very helpful in almost every project and I would recommend everyone must apply it. Whenever user delete something, soft delete it so if user done that by mistake, he will be able to recover it.

Save and display images in Binary – NodeJS

In this tutorial, you will learn, how you can save and display images in Binary in NodeJS and MongoDB.

We will also create an API that will return a binary image as a response.

Saving images in the database has many advantages over saving images in file storage.

  1. First, if you are deploying in Heroku, they do not provide persistent storage for their free tier. This means that the files uploaded on Heroku will automatically be removed after 30 minutes of inactivity.
  2. Second, migrating from one deployment platform to another is easy. Since you do not have to move all the uploaded files too. You can use mongodb.com for your MongoDB database and use this on all platforms.
  3. Third, we will be saving images in Binary format so it will take less space than saving in Base64.

Video tutorial:

The following route will save the user-uploaded image as Binary in MongoDB using NodeJS.

// npm install fs
// import file system module
const fs = require("fs")

app.post("/upload", async function (request, result) {
    // get user-uploaded file
    const image = request.files.image
  
    // reading file data
    const fileData = await fs.readFileSync(image.path)
    
    // converting to binary
    const binary = Buffer.from(fileData)
    
    // saving in database
    await db.collection("images").insertOne({
        path: binary
    })
    
    // sending response back to client
    result.send("Done")
})

Check out this tutorial if you want to know how to connect with MongoDB.

Now that the image has been saved, we will create a GET route that will return the image as a base64 string.

// set EJS as templating engine
app.set("view engine", "ejs")

app.get("/", async function (request, result) {
    // get image from collection
    const image = await db.collection("images")
        .findOne({})
        
    // variable to get base64 string
    let imageString = ""
    
    // check if document exists
    if (image != null) {
        // image.path will return binary
        // buffer() function is called on binary object
        imageString = "data:image/png;base64," + image.path.buffer.toString("base64")
    }
    
    // sending data to file "views/index.ejs"
    result.render("index", {
        image: imageString
    })
})

After that, we need to create a folder named “views” and inside it a file named “index.ejs” and write the following code in it:

<img src="<%= image %>" style="width: 100%;" />

That’s how you can save and display images in Binary in NodeJS and MongoDB.

MongoDB GridFS

In this tutorial, you will learn, how you can upload, retrieve, and delete files using MongoDB GridFS.

Video tutorial:

Upload the file to MongoDB GridFS

First, in your Node JS file, you need to create an instance of your GridFS bucket. You can create as many buckets as you want.

// include MongoDB
const mongodb = require("mongodb")

// get MongoDB client
const mongoClient = mongogb.MongoClient

// connect with MongoDB server
const client = await mongoClient.connect("mongodb://localhost:27017")

const db = client.db("mongodb_gridfs")

// create GridFS bucket instance
const bucket = new mongodb.GridFSBucket(db)
  1. First, it includes the MongoDB module.
  2. Then it gets a Mongo client object that helps in connecting with the database.
  3. Then we connect to the server.
  4. After that, we set the database.
  5. And finally, we are creating an instance of a bucket.

You can also give your bucket a name to identify.

// create GridFS bucket instance named "myBucketName"
const bucket = new mongodb.GridFSBucket(db, {
  bucketName: "myBucketName"
})

Following POST route will save the file.

// npm install fs
const fs = require("fs")

// npm install ejs
app.set("view engine", "ejs")

// npm install express-formidable
const expressFormidable = require("express-formidable")
app.use(expressFormidable())

app.post("/upload", function (request, result) {
  // get input name="file" from client side
  const file = request.files.file
  
  // set file path in MongoDB GriDFS
  // this will be saved as "filename" in "fs.files" collection
  const filePath = (new Date().getTime()) + "-" + file.name
  
  // read user uploaded file stream
  fs.createReadStream(file.path)
  
    // add GridFS bucket stream to the pipe
    // it will keep reading and saving file
    .pipe(
      bucket.openUploadStream(filePath, {
        // maximum size for each chunk (in bytes)
        chunkSizeBytes: 1048576, // 1048576 = 1 MB
        // metadata of the file
        metadata: {
          name: file.name, // file name
          size: file.size, // file size (in bytes)
          type: file.type // type of file
        }
      })
    )
    // this callback will be called when the file is done saving
    .on("finish", function () {
      result.send("File saved.")
    })
})

Now if you check in your “mongodb_gridfs” database, you will see 2 new collections.

  1. fs.files
    • This will save all uploaded files.
  2. fs.chunks
    • This will save all chunks of each file with that file ID.

Fetch all files from MongoDB GridFS

The following GET route will fetch all files uploaded to MongoDB GridFS.

app.get("/", async function (request, result) {
  const files = await bucket.find({
    // filename: "name of file" // 
  })
    .sort({
      uploadDate: -1
    })
    .toArray()
  result.render("index", {
    files: files
  })
})

Now you need to create a folder named “views” and inside that folder create a file named “index.ejs”.

Then you can loop through all files and display them in the image tag.

<% if (files) { %>
  <% files.forEach(function (file) { %>
    <p><%= file.filename %></p>
    <img src="image/<%= file.filename %>" style="width: 200px;" />
  <% }) %>
<% } %>

Right now, you will see a broken image. Now we need to create an API that will return the image as a response.

Return image as API response

app.get("/image/:filename", async function (request, result) {
  // get file name from URL
  const filename = request.params.filename
  
  // get file from GridFS bucket
  const files = await bucket.find({
    filename: filename
  })
  .toArray()
  
  // return error if file not found
  if (!files || files.length == 0) {
    return result.status(404).json({
      error: "File does not exists."
    })
  }
  
  // it will fetch the file from bucket and add it to pipe
  // result response is added in the pipe so it will keep
  // returning data to the client
  bucket.openDownloadStreamByName(filename)
    .pipe(result)
})

Now you will be able to view the image.

Delete file from MongoDB GridFS

First, you need to create a button after each file.

<% if (files) { %>
  <% files.forEach(function (file) { %>
    <p><%= file.filename %></p>
    <img src="image/<%= file.filename %>" style="width: 200px;" />
    
    <form action="/files/del" method="POST">
      <input type="hidden" name="_id" value="<%= file._id %>" />
      <button type="submit" class="btn btn-danger">Delete</button>
    </form>
  <% }) %>
<% } %>

Then you need to create an API in your Node JS file.

// 
const ObjectId = mongodb.ObjectId

app.post("/files/del", async function (request, result) {
  // get ID from data
  const _id = request.fields._id
  
  // delete file from bucket
  await bucket.delete(ObjectId(_id))
  
  // return response
  result.send("File has been deleted.")
})

This will delete the file and all its chunks from the database. So that’s all for now if you face any problem in following this, kindly do let me know.

You can learn more about the grid file system from Mongo DB’s official website.