You can buy each project separately too as well. Besides getting all the pro features of all 14 projects, we also provide additional services to them.
Secure payment
We allow users to make payments easily and securely using their bank accounts. You can contact us here and we will send you an invoice to make the payment.
Source code
Complete source code is included in all the projects. You will enjoy the pro version of each project.
Support
If you encounter any problem in installing the project or deployment, our technical support team is here. You can schedule a meeting with them and they will be able to assist you over AnyDesk or TeamViewer.
Use as a college project
If you are a student and are looking to get ready-to-go projects to learn about code and how real projects work, this will be beneficial for you. Also, Node JS projects are good learning points for students in advanced programming.
Customize as you want
Once you download the projects, you can customize them as per your needs. You can change the color theme, add new features to it, etc.
Get help in deployment
Once projects are set up in your local host, we will be here if you needed any help in deployment to the live server. For Node JS projects, we will assist you in deployment to Heroku. And for Mongo DB projects, we will help you with a deployment to mongodb.com.
MEVN stands for Mongo DB, Express JS, Vue JS, and Node JS. In this tutorial, we will teach you how you can create a picture competition web app using Node JS as a back-end server, Mongo DB as a database, and Vue JS as a front-end.
Although you do not need the assets to run this project. But if you want to have a beautiful bootstrap user interface, then you must download the required assets from this Github repo.
Setup the Project
First, you need to download and install Node JS, you can download it from their official site. Then you need to download and install Mongo DB, you can download this from their official site as well. To view the data in Mongo DB you need to download a software called “Mongo DB Compass“, you can download it from here.
Then create a new folder anywhere on your PC and rename it to “picture-competition“. Create a new folder “public” where we will place all our CSS and JS files including Bootstrap and jQuery etc. Create another folder “views” and leave it empty for now, we will place all our HTML files in it.
Node JS
Open a command prompt at the root folder of your project and run the following command:
npm init
And press “enter” for each question asked, it will set the default values. Then run the following command:
npm install express http
This will install the Express framework which we will use for routing and API calls. HTTP module will be used to start the server at a specific port. Then we need to run the following command:
sudo npm install -g nodemon
sudo is used to get admin privileges, nodemon module is used to restart the server automatically if there is any change in the server file. Then create a file named “server.js” and write the following code in it:
var express = require("express");
var app = express();
var http = require("http").createServer(app);
// [set EJS engine]
// [mongo db initialization code goes here]
http.listen(process.env.PORT || 3000, function () {
console.log("Server started");
// [mongo db connection goes here]
});
Now run the following command to start the server:
nodemon server.js
If you open the terminal, you will see the message that the server has been started.
User Registration
Login and Registration
Now we need to display a registration form, when submitted should save the record in Mongo DB. To display HTML pages in Node JS, we need to include a module named “EJS“. So run the following command to install it:
npm install ejs
Make sure to restart the server by running the nodemon server.js command. Then write the following code in your server.js by replacing the [set EJS engine] section:
Now you need to install the Mongo DB module, so run the following command:
npm install mongodb
After that, you need to write the following code in place of the [mongo db initialization code goes here] section:
var mongodb = require("mongodb");
var MongoClient = mongodb.MongoClient;
var ObjectId = mongodb.ObjectId;
After that, replace the section [mongo db connection goes here] with the following code:
MongoClient.connect("mongodb://localhost:27017", function (error, client) {
if (error) {
console.error(error);
return false;
}
var db = client.db("picture_competition");
console.log("Database connected");
// [routes goes here]
});
Check your command prompt and you will see a message that the database has been connected as well. Now we need to create a route that will be called when accessed from the browser. Write the following code in the [routes goes here] section:
app.get("/", function (request, result) {
result.render("home");
});
This will render the EJS file, so we need to create a file named “home.ejs” in your “views” folder and write the following code in it:
<%- include ("includes/header") %>
<%- include ("includes/footer") %>
Create a new folder includes inside views folder and create 2 files in it, header.ejs and footer.ejs. Following should be the content of the header.ejs file:
And following should be the content of the footer.ejs:
</body>
</html>
We will be creating multiple Vue JS apps, so right now we will create a Vue JS app for this navigation bar. Write the following lines in your footer.ejs:
<script>
var navApp = new Vue({
el: "#navApp",
data: {
baseUrl: mainURL,
login: false,
}
});
</script>
Access your project from the browser from this URL: http://localhost:3000/ and you will see the login and signup button at the top navigation bar. Because right now the user is not logged in. Click on the signup link and you will be redirected to the “404 not found” page.
Show Signup Form
Now we need to create that route in our server.js:
Then create a new file named signup.ejs inside the views folder. We will be using AJAX many times in this project, so we will place it in a separate file so we can re-use it later easily.
Create a file named app.js inside the public folder and write the following code in it:
var myApp = {
callAjax: function (url, formData, callBack = null) {
var ajax = new XMLHttpRequest();
ajax.open("POST", url, true);
ajax.onreadystatechange = function () {
if (this.readyState == 4) {
if (this.status == 200) {
if (callBack != null) {
callBack(this.responseText);
}
}
if (this.status == 500) {
console.log(this.responseText);
}
}
};
ajax.send(formData);
}
};
Then in your header.ejs you need to include that file as Javascript.
<script src="/app.js"></script>
We need to display an alert message when the user is signed up or when there is an error during sign-up. We will be using the Sweetalert library to display alerts. Include that in your header.ejs:
<script src="/sweetalert.min.js"></script>
Finally, we will write the following code in our signup.ejs:
<%- include ("includes/header") %>
<div class="container margin-container" id="registerApp">
<div class="row">
<div class="offset-md-3 col-md-6">
<h2 class="text-center">Register</h2>
<form method="POST" v-bind:action="baseUrl + '/signup'" v-on:submit.prevent="doRegister">
<div class="form-group">
<label>Name</label>
<input type="text" name="name" class="form-control" required />
</div>
<div class="form-group">
<label>Email</label>
<input type="email" name="email" class="form-control" required />
</div>
<div class="form-group">
<label>Password</label>
<input type="password" name="password" class="form-control" required />
</div>
<input type="submit" name="submit" value="Register" class="btn btn-primary" />
</form>
</div>
</div>
</div>
<script>
var registerApp = new Vue({
el: "#registerApp",
data: {
baseUrl: mainURL
},
methods: {
doRegister: async function () {
const form = event.target;
// disable the submit button and show "Loading..." text
form.submit.setAttribute("disabled", "disabled");
form.submit.value = "Loading...";
var formData = new FormData(form);
myApp.callAjax(form.getAttribute("action"), formData, function (response) {
// convert the JSON string into Javascript object
var response = JSON.parse(response);
// enable the submit button
form.submit.removeAttribute("disabled");
form.submit.value = "Register";
// if the user is created, then redirect to login
if (response.status == "success") {
swal("Registered", response.message, "success");
} else {
swal("Error", response.message, "error");
}
});
}
}
});
</script>
<%- include ("includes/footer") %>
Refresh the page and you will see a signup form. But nothing happens when you click the submit button, it is because we need to create an API for signup.
Saving the Data in Mongo DB
We will be sending AJAX requests with a form data object in this project. To parse that form data object on the server-side, we need to install a module named express-formidable.
And we need to encrypt plain text password fields into a hashed string before saving them in Mongo DB. So we need to install a module named bcrypt.
After the user successfully registered, we need to send a verification link to his email address to verify the account. He will only be able to log in if the account is verified. To send an email we will be using a module named nodemailer.
Note: Please enable the “less secure apps” option from your google account from this page.
Then open your command prompt and run the following commands at the root of your project folder to install the above-discussed modules:
npm install express-formidable bcrypt nodemailer
Then open your server.js file and include the above-installed module at the top:
Enter your correct email and password of your Google account of whom you have enabled “less secure apps”. Then create a post route for “signup” by chaining it with GET route we created earlier.
// route for signup requests
app.route("/signup")
// get request accessed from browser
.get(function (request, result) {
// render signup.ejs file inside "views" folder
result.render("signup");
})
// post request called from AJAX
.post(async function (request, result) {
// get values from signup form
var name = request.fields.name;
var email = request.fields.email;
var password = request.fields.password;
var verificationToken = new Date().getTime();
var createdAt = new Date().getTime();
// check if email already exists
var user = await db.collection("users").findOne({
"email": email
});
if (user != null) {
result.json({
"status": "error",
"message": "Email already exists."
});
return true;
}
// encrypt the password
bcrypt.hash(password, 10, async function (error, hash) {
// insert in database
await db.collection("users").insertOne({
"name": name,
"email": email,
"password": hash,
"picture": "",
"accessToken": "",
"notifications": [],
"bio": "",
"dob": "",
"country": "",
"phone": "",
"website": "",
"twitter": "",
"facebook": "",
"googlePlus": "",
"linkedIn": "",
"instagram": "",
"resetToken": "",
"isVerified": false,
"verificationToken": verificationToken,
"createdAt": createdAt
});
var transporter = nodemailer.createTransport(nodemailerObject);
var html = "Please verify your account by click the following link: <br><br> <a href='" + mainURL + "/verifyEmail/" + email + "/" + verificationToken + "'>Confirm Email</a> <br><br> Thank you.";
transporter.sendMail({
from: nodemailerFrom,
to: email,
subject: "Email Verification",
text: html,
html: html
}, function (error, info) {
if (error) {
console.error(error);
} else {
console.log("Email sent: " + info.response);
}
// send the response back to client
result.json({
"status": "success",
"message": "Signed up successfully. Kindly check your email to verify your account."
});
});
});
});
Refresh the browser and try to signup. Fill the signup form and hit submit. You will see a Sweetalert that an account has been created and you will also receive an email of account confirmation.
Email Verification
Email Verification
Now we need to create a route to verify the email address because without this, the user will not be able to log in. So we just need to create a GET route in our server.js:
app.get("/verifyEmail/:email/:verificationToken", async function (request, result) {
const email = request.params.email;
const verificationToken = request.params.verificationToken;
var user = await db.collection("users").findOne({
$and: [{
"email": email,
}, {
"verificationToken": parseInt(verificationToken)
}]
});
if (user == null) {
result.render("verify-email", {
"status": "error",
"message": "Email does not exists. Or verification link is expired."
});
return false;
}
await db.collection("users").findOneAndUpdate({
$and: [{
"email": email,
}, {
"verificationToken": parseInt(verificationToken)
}]
}, {
$set: {
"verificationToken": "",
"isVerified": true
}
});
result.render("verify-email", {
"status": "success",
"message": "Account has been verified. Please try login."
});
});
If the account is confirmed successfully or if there is an error in the confirmation like we will add the expiry time to the link. So the user has to resend a verification email if a certain time is passed. In either case, we need to create a new file verify-email.ejs inside your views folder to display a success or an error message.
Click on the link you receive in your provided email address, you will see a success message using a Bootstrap alert if the account is confirmed successfully.
In this article, we are going to show you, how you can change title text of an HTML page using simple Javascript. We will show a real-time notification to your users in their title bar using PHP, MySQL, and Node JS. You can show a notification in a title bar with just PHP and MySQL. But the “real-time” effect can only be achieved with Node JS.
We can change title text using Javascript’s document.title object, but we will show you with a real-world example.
What we will do:
Save notification in the database
Display counter in the title bar
Show all unread notifications to the user
Mark notification as read
Make notifications real-time
1. Save notifications in the MySQL database
First, we need to save the notifications in the database. Notifications can be of any type, for example, “a new message is received” or “your account has been activated” etc. We will create a new file named “send-notification.php“.
<?php
// connect with database
$conn = new PDO("mysql:host=localhost;dbname=test", "root", "");
// create table if not exists
$sql = "CREATE TABLE IF NOT EXISTS notifications (
id INTEGER NOT NULL PRIMARY KEY AUTO_INCREMENT,
user_id INTEGER,
message TEXT,
is_read BOOLEAN,
created_at DATETIME
)";
// prepare the query
$statement = $conn->prepare($sql);
// execute the statement
$statement->execute();
// user who needs to receive the notification
$user_id = 565; // replace this your user ID
$message = "You have received a new message " . time() . ".";
$is_read = 0;
// insert in notifications table
$sql = "INSERT INTO notifications (user_id, message, is_read, created_at) VALUES (?, ?, ?, NOW())";
$statement = $conn->prepare($sql);
$statement->execute([
$user_id, $message, $is_read
]);
Comments have been added with each line for an explanation. Run the above code and check your database, you will see a new table named “notifications” and also a new row is inserted in that table. Now we need to display this on the user side.
2. Display the counter in the title bar
We just need to fetch the total number of unread notifications from the database and display them in the user’s title bar. Create a new file named “index.php” and it will have the following code:
<?php
// start session and login the user
session_start();
// you might be saving the session during login,
// I am hard-coding the value for testing purpose
$_SESSION["user_id"] = 565;
// connect with database
$conn = new PDO("mysql:host=localhost;dbname=test", "root", "");
// get number of unread notifications
$sql = "SELECT COUNT(*) AS total_unread_notifications FROM notifications WHERE user_id = ? AND is_read = 0";
$statement = $conn->prepare($sql);
$statement->execute([
$_SESSION["user_id"]
]);
$row = $statement->fetch();
$total_unread_notifications = $row["total_unread_notifications"];
Then we need to fetch this variable in Javascript and prepend it in the title bar. First, we need to create a title tag and a hidden input field where we will put the above variable value.
<!-- title where notification number will be displayed -->
<title>My Website</title>
<!-- save variables in hidden input field to access in Javascript -->
<input type="hidden" id="total-unread-notifications" value="<?php echo $total_unread_notifications; ?>" />
Then we need to get this input field value in Javascript and render it in the title tag. We will create a separate Javascript function to render the title bar because we will need it in multiple situations when:
The page is loaded
There is a new notification (increment)
A notification is read by the user (decrement)
<script>
// get variables in Javascript
var totalUnreadNotifications = document.getElementById("total-unread-notifications").value;
totalUnreadNotifications = parseInt(totalUnreadNotifications);
// show count in title bar
showTitleBarNotifications();
function showTitleBarNotifications() {
// pattern to check if there is any counter number at the start of title bar
var pattern = /^\(\d+\)/;
if (totalUnreadNotifications == 0) {
document.title = document.title.replace(pattern, "");
return;
}
if (pattern.test(document.title)) {
// update the counter
document.title = document.title.replace(pattern, "(" + totalUnreadNotifications + ")");
} else {
// prepend the counter
document.title = "(" + totalUnreadNotifications + ") " + document.title;
}
}
</script>
This will first check if the value of variable totalUnreadNotifications is greater than 0. If it is 0, then we will not show any number at all (not even 0). Then we will check if there is already any value in the title bar. If there is any value, then we will simply increment the counter value and re-render the title bar.
If there isn’t any value already in the title bar, then we will simply prepend the counter before the title tag content.
Here, the document.title is the only line that change title text.
This does change title text because initially, it was only “My Website”. After a notification is fetched from database, it prepends the value in the <title> tag.
At this point, you are seeing the number of unread notifications in your title bar. Now you need to find a way to mark notifications as “read” and decrement the counter.
3. Show all unread notifications to the user
To mark notifications as “read”, we first must show all the notifications to the user which when clicked will be marked as “read”. After that, the counter will be decremented and the title tag will be re-rendered. We will create a new file named “notifications.php” to display all notifications to the user.
<?php
// start session
session_start();
// connect with database
$conn = new PDO("mysql:host=localhost;dbname=test", "root", "");
// get all notifications sorting by unread goes first
$sql = "SELECT * FROM notifications WHERE user_id = ? ORDER BY is_read ASC";
$statement = $conn->prepare($sql);
$statement->execute([
$_SESSION["user_id"]
]);
$notifications = $statement->fetchAll();
This will fetch all the notifications of the logged-in user. Now we need to display them in an HTML table. We will simply by displaying a notification message and a button to mark that notification as read.
<!-- show all notifications in a table -->
<table>
<tr>
<th>Message</th>
<th>Action</th>
</tr>
<?php foreach ($notifications as $notification): ?>
<tr>
<td><?php echo $notification['message']; ?></td>
<td>
<!-- show 'read' button only if the notification is un-read -->
<?php if (!$notification['is_read']): ?>
<form onsubmit="return markAsRead();">
<input type="hidden" name="id" value="<?php echo $notification['id']; ?>" />
<input type="hidden" name="user_id" value="<?php echo $notification['user_id']; ?>" />
<input type="submit" value="Read" />
</form>
<?php endif; ?>
</td>
</tr>
<?php endforeach; ?>
</table>
If you refresh the page now, you will see a list of all notifications with the button “read”. On clicking that button, we need to call an AJAX request to the server to mark this notification as “read” in the MySQL database. We will be using Vanilla JS to do that, no jQuery or any other external library is being used.
Now we need to create this Javascript function that will actually send the AJAX request. We will be using AJAX because there will be a lot of notifications and it will not be a good idea to keep refreshing the page for each notification to be marked as read.
<script>
// when the read button is clicked
function markAsRead() {
// prevent the form from submitting
event.preventDefault();
// get the form node
var form = event.target;
// create AJAX object
var ajax = new XMLHttpRequest();
// set method and URL of request
ajax.open("POST", "read-notification.php", true);
// when the status of request changes
ajax.onreadystatechange = function () {
// when the response is received from server
if (this.readyState == 4) {
// if the response is successful
if (this.status == 200) {
// convert the JSON string into Javascript object
var data = JSON.parse(this.responseText);
console.log(data);
// if there is no error
if (data.status == "success") {
// remove the 'read' button
form.remove();
// [emit read notification event here]
}
}
}
};
// create form data object with the form
var formData = new FormData(form);
// send the AJAX request with the form data
ajax.send(formData);
}
</script>
This will prevent the form from redirecting and calling our Javascript code. After the notification is successfully marked as read from the server, we will remove the “read” button. The last thing we need is a server-side PHP file that will handle this request.
4. Mark notification as read
Now we need to create a new file named “read-notification.php” that will mark this notification as “read”. In this file, we will also check the user_id along with the notification ID to make sure that the notification we are marking is “read” is actually sent to the logged-in user.
<?php
// start the session
session_start();
// connect with database
$conn = new PDO("mysql:host=localhost;dbname=test", "root", "");
// get ID from AJAX
$id = $_POST["id"];
// mark notification as read
$sql = "UPDATE notifications SET is_read = 1 WHERE id = ? AND user_id = ?";
$statement = $conn->prepare($sql);
$statement->execute([
$id,
$_SESSION["user_id"]
]);
// send the response back to client
echo json_encode([
"status" => "success"
]);
exit();
This file will first check that the notification ID sent from AJAX actually refers to the logged-in user. Because someone can try to tamper with the client-side code and mark any other user’s notification as “read”. So we must do server-side validation like above.
At this point, if you run the code now, you will be able to see all the notifications in an HTML table with a “Read” button. When clicked, will mark the notification as read and also will remove the “read” button.
If you refresh the “index.php” file, you will now see the counter in the title bar will be decremented. One more thing you can do is to make it real-time, so when there is any notification added, the counter will be incremented in the title bar. And also when the notification is read by the user, then the counter should be decremented automatically.
5. Make notifications real-time
Making these notifications in real-time is crucial. You might have used the WhatsApp Web where you will see that the counter in the title bar automatically gets incremented and decremented based on the number of new notifications you received and the notifications have you read.
Now we need to learn how you can implement this functionality in your project. We will be using the Socket IO JS library for this and we will also create a small Node JS server for this purpose. You can download Socket IO JS from their official site.
After downloading, you need to paste the JS file into your project. You also need to download and install Node JS in your system, you can download it from here.
Setting up Node JS server
After Node JS installation, open the command prompt or terminal in your project root directory and run the following command:
npm init
This will ask a couple of questions, you can press “enter” for all questions and it will automatically set the default answer. Then we need to install the required modules for this feature. Run the following command in your terminal:
npm install express http socket.io
This will install the Express, HTTP, and Socket IO module in your Node JS app. To start the server, we need to install another module globally named “nodemon“. So again, run the following command in your terminal:
npm install -g nodemon
Typically when you made changes in your code, you need to manually restart the Node JS server. But it slows down the process during development. So this module automatically restarts the server if there is any change in the server file.
Create a new file named “server.js“, this will be our Node JS server file. Paste the following code in that file, we will explain this in the next step:
// initialize express server
var express = require("express");
var app = express();
// create http server from express instance
var http = require("http").createServer(app);
// include socket IO
var socketIO = require("socket.io")(http, {
cors: {
origin: ["http://localhost"]
}
});
// start the HTTP server at port 3000
http.listen(process.env.PORT || 3000, function () {
console.log("Server started running...");
// an array to save all connected users IDs
var users = [];
// called when the io() is called from client
socketIO.on("connection", function (socket) {
// called manually from client to connect the user with server
socket.on("connected", function (id) {
users[id] = socket.id;
});
});
});
In this file we are:
Initializing the Express framework and also HTTP module.
Including the Socket IO module. You can place your own server URL in the CORS origin array.
Then we are starting the server at port 3000. We use process.env.PORT on deployment.
When a new user is connected with a server, a unique socket ID is generated. We are saving each user’s socket ID in a local array along with his ID from the database.
Now is the time to start the server. Run the following command in your terminal to start the server.
nodemon server.js
If you check the terminal, you will see a message “Server started running…” this means that you are ready to send connect your client-side with this server.
Increment counter on new notification
Whenever a new notification is sent to the server, we need to automatically increment the counter in the title bar. We have already created a function in “index.php” that displays the counter in the title bar.
To add this feature we need to perform the following tasks:
Connect socket IO JS on the “send-notification.php” file.
Then emit (send) the notification event to the Node JS server along with the ID of the user.
On the Node JS server-side, listen to that event and emit the event to the relative user.
Connect socket IO JS on the “index.php” file.
Listener for that new notification event.
When that event is received, increment the counter and re-render the title.
The following code goes in the “send-notification.php” file after the notification is inserted in the database:
<!-- save user id -->
<input type="hidden" id="user-id" value="<?php echo $user_id; ?>" />
<!-- include socket IO JS -->
<script src="socket.io.js"></script>
<script>
// connect with Node JS server
var socketIO = io("http://localhost:3000");
// get user ID
var userId = document.getElementById("user-id").value;
// send notification to the server
socketIO.emit("newNotification", userId);
</script>
We are already creating the $user_id variable in the first step. Then we are simply including the socket IO JS library and connect with the Node JS server. Getting the user ID from the hidden input field. And finally emitting an event to the server with a user ID.
In your “server.js” listen to this event and emit the event to the user with the same ID.
// when a new notification is received
socket.on("newNotification", function (userId) {
// send notification to the selected user
socketIO.to(users[userId]).emit("newNotification", userId);
});
The event has been dispatched from the server to the client. Now the client must listen to this event and when received should increment the counter and re-render the title. The following code goes in your “index.php“:
<!-- include socket IO JS -->
<script src="socket.io.js"></script>
<script>
// connect with Node JS server
var socketIO = io("http://localhost:3000");
// connect user with Node JS server
var userId = document.getElementById("user-id").value;
socketIO.emit("connected", userId);
// when a new notification is received
socketIO.on("newNotification", function (data) {
totalUnreadNotifications++;
showTitleBarNotifications();
});
</script>
Open “index.php” and “send-notification.php” in separate tabs and refresh both of these pages. Every time you refresh the send notification page, you will see the title of the index file gets incremented. Now we need to do the same for reading notifications, except that the title bar will be decremented whenever a notification is marked as read.
Decrement counter on reading notification
In “notifications.php” first, we need to include the socket IO JS library and connect with the server.
<!-- include socket IO JS -->
<script src="socket.io.js"></script>
<script>
// connect with Node JS server
var socketIO = io("http://localhost:3000");
</script>
Following code goes in the [emit read notification event here] section:
// send notification to the server
socketIO.emit("notificationRead", form.user_id.value);
Now we need to create a listener for this in our “server.js” file.
socket.on("notificationRead", function (userId) {
socketIO.to(users[userId]).emit("notificationRead", userId);
});
Similarly, you need to create a listener on the user side too in the index.php file.
socketIO.on("notificationRead", function (data) {
totalUnreadNotifications--;
showTitleBarNotifications();
});
This also change title text, but this time it decrements the counter value.
Run the code now and refresh both of your browsers. And try to read some notifications, as soon as you read them, you will see that the counter in the title bar will be decremented too.
So that’s how you can change title text of an HTML page using vanilla Javascript.
Conclusion
So that’s how you can implement a real-time notification system in your website with an unread notification counter in the title bar using simple PHP, Javascript, and Node JS. There is no PHP or Javascript framework used in this tutorial, so you can work with it with any framework in your existing project.
That’s how you can change the title text of the browser tab. If you face any problems in following this, kindly do let me know in the comments section below.
Picture Competition is a mobile responsive real-time web application developed in Node JS and Mongo DB. Its frontend is designed in Bootstrap and Vue JS. You can create competition between 2 people and the others can vote on the person based on looks or skills etc.
It uses login authentication using JWT (JsonWebToken). It does not use the browser session due to the following reasons:
Sessions are destroyed once your app is restarted from terminal.
Sessions are not linked with the server, they are stored in browser only.
The benefit of using JWT is that you can always log out a user from your Mongo DB. Just go to the “users” collection and find the specific user. Then empty the “accessToken” field of that user’s document.
Login and Registration
2. Create Competitions
Registered users can create competitions between 2 users. You can enter name and upload 1 picture of each competitor. There is no limit in the number of competitions to create, you can create as many as you want. You can view your created competitions on your “My Competitions” page.
As soon as you create a competition, it will automatically be displayed to all the users as well as to the admin in real-time. Users do not have to refresh the page to see new competitions. It uses Socket IO to display data in real-time.
Create Competition
3. Search
Users can search competitions by the name of competitors. Data is filtered and rendered using Vue JS. There is no need to press the enter key, the data is filtered as soon as the user starts typing in the search box.
Search and Sort
4. Sort
Users can also sort the competitions by any of the following for sorting functions:
Highest votes to lowest.
Lowest votes to highest.
Oldest to newest.
Newest to oldest (default).
5. Vote on Competition
Logged-in users can vote on the competition. You can either vote on one of the competitors. Once the vote is cast on one competitor, it cannot be removed. Also, you can cast your vote on only one of the competitors, not on both of them. It is also real-time, as soon as the vote is cast, it will automatically be displayed to all the users and the counter is incremented. The counter displays the number of votes cast on each competitor.
Votes
6. Delete Competition
Competitions can only be deleted by either of the users who created the competition, or by the admin. Once the competition is deleted, all the uploaded images will be deleted too. As soon as the competition is deleted, it will automatically be removed from all the other users too, without having them refresh the page.
7. Realtime Update using Sockets
Sockets are used for real-time communication. Instead of fetching the data from the server after regular intervals, sockets attach listeners to the client-side. Listeners are listening to the events sent from the server. The server will emit the event and the client will listen to that event and respond accordingly. In this project, sockets are used for the following features:
When competition is created.
When competition is deleted.
To increase the counter after vote is casted to the competition.
Notifications.
8. Notifications
When a competition is deleted by the admin, the admin will write the reason for the deletion. Thus, a notification will be sent to the user along with the reason why his competition was removed. By default, notification status is “unread” and they are highlighted. As soon as the user clicks on any of the notifications, that notification will be marked as “read” and it will no longer be highlighted.
Notifications
9. Load More Button
When the data in the database increases, it is not feasible to load all the data in a single AJAX request. So a “load more” button is created to solve this problem. For example, 10 records are fetched in the first AJAX request. The next 10 records will be fetched when the “load more” button is clicked, and so on.
Load More
10. Admin Panel
Admin panel is created so you (administrator) can delete any competition you find offensive. The default email and password of admin are:
email = admin@gmail.com password = admin
11. Manage Competitions
Admin can delete competitions that he finds offensive. However, the admin must give the reason why that competition is deleted. A notification will be sent to the user who created that competition and he will be able to view it from the top navigation bar.
12. Reset Password
Now you will be able to reset your password if you ever forgot. You just need to enter your email address and an email will be sent to you with a link to reset the password. We are using the nodemailer module to send an email.
Forgot PasswordReset Password
13. Email Verification
When a new user registers, we are sending a verification email to the user’s entered email address. The user will not be able to log in until he verifies his email address. When a user clicks the link on his email address, he will receive a message that says that his account is verified. Then he will be able to log in successfully.
Email Verification
14. SMTP Configurations from Admin Panel
To send an email, you will need an SMTP server. Every SMTP server requires some configurations to set up that include, host, port, email, and password. You can write these values directly hardcoded in your code, but to update these values in the future, you have to find these values in the code and update them.
In this project, you can set these configurations directly from the admin panel. Once the values are set, new emails will be sent using the new configurations.
SMTP configurations from admin panel
15. Adult Image Validation
This is a must-have feature if you are creating a website that allows users to upload pictures and they will be seen to the world. Anyone can upload an image that contains adult content, and it will not be good for your business. So when the user is uploading pictures while creating competition, the system will automatically check if the image is safe to upload.
If the image is an adult image, then an error will be shown to the user and it will not be uploaded.
16. Admin Panel Stats
Admin can see total users, total competitions, and total votes cast so far. They are also real-time, so when a new user is registered, or new competition is created, or event a new vote is cast, it will automatically be incremented here.
Also, when competition is deleted, the number will automatically get decremented as well, without having the admin refresh the page.
Admin Panel Stats
17. Real-time Comments
Users can comment on each competition. And they are also real-time as well. Once a new comment is added, it will immediately be displayed to all the other users as well. They do not have to refresh the page to see new comments.
Real-time Comments
18. User Profile
Users can now update their names and profile pictures. We are using the fs (file system) module to upload the picture. User can also add their bio, date of birth, country, and social media links. Media links include Facebook, Instagram, google plus, Twitter, and LinkedIn.
User can also change their account password. In order to change the password, the user must enter the current password. The new password should be entered twice for confirmation.
User Profile
19. Free Customer Support
This is not a feature of the project, but it is a free service provided for the pro version only. That means if you find any difficulty in installing or configuring the project, we will help you install it. Also, if you encounter any error or a bug in the released version, then it can be fixed too.
These are all the features we have right now in the picture competition web app. We are open to more ideas. If you have more ideas to add, kindly do let us know.
A Financial Ledger script is created in Node JS, Mongo DB, and Vue JS. A financial ledger is used to track your daily finances, for example, your income, your expenses, etc. This script allows you to track your total income, total spending, and your current total balance. They are referred to as “stats” in the script. You can debit the entry by entering the amount that is negative. And you can credit an entry by simply writing its amount.
We have used the HTML and CSS code from this codepen. We have made it dynamic by using the latest technologies like, Vue JS for front-end, Node JS for back-end, and Mongo DB as the database. It has complete CRUD (Create, Read, Update, Delete) operation. It has the following features:
Insert data in Mongo DB using Node JS.
Fetch all documents from Mongo DB.
Update specific document in Mongo DB.
Delete documents from Mongo DB.
Case-insensitive search in Mongo DB documents.
Search by sub-string from Mongo DB document’s keys.
Realtime data insert.
Real-time update.
Realtime deletes.
“Load more” capability.
Datetimepicker library.
EJS templating engine.
Express-formidable for handling form fields.
Skip, limit and sort in Mongo DB.
Video tutorial:
1. Insert Data in Mongo DB using Node JS
An interface that allows you to enter the values (time, description, and amount), created in Vue JS. It binds those input field values to the Vue JS model. To enter the time, we are using a library datetimepicker. We are calling an AJAX request to the Node JS server with the input field values as FormData object when the form submits. That will simply insert a new document in Mongo DB collection “entries”. When the data is inserted, it is also inserted in the Vue JS array. This allows the new data to be appended automatically without having to refresh the page. The stat values automatically are updated based on if the entry is debit or credit.
2. Fetch All Documents from Mongo DB
An AJAX request is sent from the client to view all the data from the database. The server will return all the documents sorting from latest to oldest i.e. newest records will be shown first. The sorting is performed based on the date field selected during insertion. When the data is returned, it is concatenated in the Vue JS array that allows the user to view the data when the page loads. When the data is fetched, the stats are also updated automatically.
3. Update Specific Document in Mongo DB
To update specific documents, we are using Mongo DB auto-generated ObjectId field “_id”. This allows you to search the document, then we can perform the update query. To update the document, we are using HTML’s contenteditable attribute that allows you to edit the HTML node innerHTML by just typing in it. When the content of that div changes, we are sending an AJAX request to the Node JS server that performs the update query. If you update the amount value, then the stats will also get a change in real-time.
4. Delete Documents from Mongo DB
Deleting an entry will remove the document from the Mongo DB collection too. The row will be removed and the stats will be updated too.
5. Case insensitive Search Mongo DB
You can search the entries by description you have put during the insertion, or you can search the entries by either they are debited or credited. For example, you can write “debit” in the search box and it will show all the entries that are debited. The same goes for the “credit” search query.
6. Sub-string Search Mongo DB
While searching by description, you do not need to know the exact words. You can type the part of the description you have remembered and it will search the record anyway.
7. Realtime Mongo DB Data Insert
When a new entry is added, it is automatically prepended in the list using Vue JS. You do not have to refresh the page to view the new entries.
8. Realtime Update in Mongo DB
The same goes for the updation. You can update the entry’s description or the amount by simply typing in its box. When you start typing it automatically gets updated in the Mongo DB too. Also, the stats values also get updated if there is any change in the amount.
9. Realtime Delete in Mongo DB
With each entry, there is a minus (-) sign. On clicking that button, it will remove the entry from the list and also from the Mongo DB. The stats also gets updated accordingly.
10. Load More Capability
In the real-world, when you are creating a financial ledger, there will be hundreds of entries. So loading all entries when the page loads will slow down your application. So what we do is, load a few entries when the page loads. Then we show a button called “Load more”. Upon clicking that button, we will fetch the next entries from Mongo DB and so on. We will be using AJAX to fetch more entries.
11. Datetimepicker Javascript
In the financial ledger, it is important to enter the date the entry was added. So on the web, we have a library called datetimepicker by XDSoft. We are using this library to enter dates and times easily.
12. EJS Templating Engine in Node JS
In Node JS, to render files, there is an engine called EJS. It is used to render HTML files. The rendered files will have an extension “.ejs” instead of “.html” or “.php”.
13. Express Formidable Module in Node JS
In Node JS, to handle FormData object sent using AJAX, we are using a Node JS module named express-formidable. As the complete app is in Javascript, so there are a lot of AJAX requests in this app. Each AJAX request will send a FormData object to send values that the server will process.
14. Skip, Limit and Sort in Mongo DB
While fetching the records from Mongo DB, you can skip the records that are already been displayed to the user. Similarly, to load the data faster, you can limit the number of records to be fetched from Mongo DB in one request. To display the latest entries added, you can sort the records fetched from Mongo DB.
Sockets are used for real-time communication. They are now being used in chat apps, team collaboration tools, and many more. Socket IO emit events to the receivers and the receivers are constantly listening to that event. When the event is received on the client-side, they can perform the necessary action. You can attach as many event listeners as you want and perform different actions for each event.
Users are connected with a Node JS server using a client-side library called Socket IO. Users can also join the room which will be helpful if you are creating a group chat app. There are 4 ways in which socket events are fired.
Send event to all connected users, including the sender.
Send event to all users, except the sender.
Emit event to all users in a room.
Send event to specific users.
In this tutorial, we will be covering the 4th part i.e. send socket events to specific users.
Video tutorial:
Problem
Suppose you have a chat app where you want 2 people to have a private chat. Now you want to have a real-time effect i.e. to show the new messages without having to refresh the page. This requires sockets that send the data in real-time. And we can show the data on the client-side in real-time too. Now when a sender sends a message to a specific user, we need to send the socket event to that specific user only.
Solution
We will create a simple script that allows us to send events to a specific user only. You can then integrate and customize that logic in your project. First, you need to download and install Node JS. You also need to download the Socket IO JS client-side library. We will have a simple database from where we can show all the users in a list, with a button to send an event to that user only. So we need to create a database with a simple users table, you can use your own database as well.
Database
In your phpMyAdmin, create a database named “send_socket_event_to_specific_users”. In that database, create a users table:
CREATE TABLE `users` (
`id` int(11) NOT NULL PRIMARY KEY AUTO_INCREMENT,
`name` text NOT NULL
);
Add few rows in that table so we can show them in a list or a table.
It will show all users in a table with a button to send message. Now when the page loads, we need to get the ID of the user, you can also get it from PHP sessions.
Include Socket IO library
Before that, we need to include the Socket IO JS library. You can download it from here.
<script src="socket.io.js"></script>
<script>
var userId = prompt("Enter user ID");
var socketIO = io("http://localhost:3000");
socketIO.emit("connected", userId);
</script>
It will store your ID in userId variable. And it will connect with Node JS server and emit an event “connected” with your ID.
Now we need to create a simple Node server. Create an empty folder and create a file named “server.js” in it. Then open the CMD in that folder and run the following commands one-by-one:
var express = require("express");
var app = express();
var http = require("http").createServer(app);
var socketIO = require("socket.io")(http, {
cors: {
origin: "*"
}
});
var users = [];
socketIO.on("connection", function (socket) {
socket.on("connected", function (userId) {
users[userId] = socket.id;
});
// socket.on("sendEvent") goes here
});
http.listen(process.env.PORT || 3000, function () {
console.log("Server is started.");
});
This will start the server at port 3000, creates a users array and store all connected user’s socket ID in it.
Send event using socket IO emit function
Back in index.php, we need to create a JS function to send the event when the user click the “Send message” button:
function sendEvent(form) {
event.preventDefault();
var message = prompt("Enter message");
socketIO.emit("sendEvent", {
"myId": userId,
"userId": form.id.value,
"message": message
});
}
Now in server.js, we need to listen to that event and send the message to that user only. But before that, we need to include mysql module because the user’s names are stored in mysql database. At the top of your server.js:
var mysql = require("mysql");
var connection = mysql.createConnection({
host: "localhost",
port: 3306,
user: "root",
password: "",
database: "send_socket_event_to_specific_users"
});
connection.connect(function (error) {
console.log("Database connected: " + error);
});
And after the socket connected event:
socket.on("sendEvent", async function (data) {
connection.query("SELECT * FROM users WHERE id = " + data.userId, function (error, receiver) {
if (receiver != null) {
if (receiver.length > 0) {
connection.query("SELECT * FROM users WHERE id = " + data.myId, function (error, sender) {
if (sender.length > 0) {
var message = "New message received from: " + sender[0].name + ". Message: " + data.message;
socketIO.to(users[receiver[0].id]).emit("messageReceived", message);
}
});
}
}
});
});
This will search the sender and receiver by ID, and emit the event to the receiver with the name of the sender.
Listen to socket IO events
Now we need to listen to that event in our index.php and show a message in a list when that event is received. First, create a ul where all messages will be displayed:
<ul id="messages"></ul>
Then attach that event in JS:
socketIO.on("messageReceived", function (data) {
var html = "<li>" + data + "</li>";
document.getElementById("messages").innerHTML = html + document.getElementById("messages").innerHTML;
});
So that’s how you can use the socket IO emit function to send the event to a specific user only.
Check out realtime chat app tutorial using socket IO.
In this article, we are going to create a web crawler using Node JS and Mongo DB. It will take a URL as an input and fetch all the anchor tags, headings, and paragraphs. You can add more features to it if you want.
Requirements
Make sure you have the following things installed in your system:
Node JS
Mongo DB
Code Editor (Sublime Text etc.)
Video tutorial:
Setup the Project
First, create an empty folder anywhere in your system. Create a file named server.js in that folder. Open CMD in that folder by running the following command:
cd "path_of_your_folder"
We are going to need multiple modules for this web crawler. So, install them from the command:
Now we explain the reason for the installation of the above modules.
express framework is used for routing.
http is used to run HTTP requests.
ejs is a template engine used for rendering HTML files.
socket.io is used for realtime communication.
request is used to fetch content of web page.
cheerio is used for jQuery DOM manipulation.
express-formidable to get values from FormData object.
mongodb will be our database.
htmlspecialchars is used to convert HTML tags into entities.
node-html-parser to convert the HTML string into DOM nodes.
After all the modules are installed, run the following command to start the server:
npm install -g nodemon
nodemon server.js
Start the server
Open your server.js and write the following code in it to start the server at port 3000.
var express = require("express");
var app = express();
var http = require("http").createServer(app);
http.listen(3000, function () {
console.log("Server started running at port: 3000");
});
Now your project will be up and running at http://localhost:3000/
Connect Node JS with Mongo DB
To connect Node JS with Mongo DB, first, we need to create an instance of Mongo DB client in our server.js. Place following lines before http.listen function.
var mongodb = require("mongodb");
var mongoClient = mongodb.MongoClient;
var ObjectID = mongodb.ObjectID;
var database = null;
Now write the following code inside the http.listen callback function.
If you check your CMD now, you will see the message “Database connected”.
Crawl the web page
Now we need to create a form to get the URL as an input. So first we will tell our express app that we will be using EJS as our templating engine. And all our CSS and JS files will be inside the public folder. Place following lines before http.listen function.
Now create 2 folders at the root of your project, “public” and “views”. Download the latest jQuery, Bootstrap, DataTable, and Socket IO libraries and placed their files inside the public folder. Create a new file named index.ejs inside views folder. Create a GET route in our server.js when the Mongo DB is connected.
app.get("/", async function (request, result) {
result.render("index");
});
If you access your project from the browser now, you will see an empty screen. Open your index.ejs and write the following code in it:
You will now see a simple form with an input field and a submit button. In that input field, you can enter the URL of the page you wanted to crawl. Now we need to create a Javascript function that will be called when the form is submitted. In that function, we will call an AJAX request to the Node JS server.
<script>
function crawlPage(form) {
var ajax = new XMLHttpRequest();
ajax.open("POST", "/crawl-page", true);
ajax.onreadystatechange = function () {
if (this.readyState == 4) {
if (this.status == 200) {
// console.log(this.responseText);
var data = JSON.parse(this.responseText);
// console.log(data);
}
}
};
var formData = new FormData(form);
ajax.send(formData);
return false;
}
</script>
Get the web page content
To fetch the content of the web page, first, we will use express-formidable as our middleware. Also, we will require the modules required to read the web page and convert its HTML into DOM nodes. Write the following lines before http.listen function.
After that, we will create a POST route to crawl the web page.
app.post("/crawl-page", async function (request, result) {
var url = request.fields.url;
crawlPage(url);
result.json({
"status": "success",
"message": "Page has been crawled",
"url": url
});
});
Our web crawler runs in a separate function to crawl the web page. Then we will create the functions to crawl the web page and save its content in Mongo DB. Write the following functions at the top of your server.js file.
function getTagContent(querySelector, content, pageUrl) {
var tags = content.querySelectorAll(querySelector);
var innerHTMLs = [];
for (var a = 0; a < tags.length; a++) {
var content = "";
var anchorTag = tags[a].querySelector("a");
if (anchorTag != null) {
content = anchorTag.innerHTML;
} else {
content = tags[a].innerHTML;
}
content = content.replace(/\s+/g,' ').trim();
if (content.length > 0) {
innerHTMLs.push(content);
}
}
return innerHTMLs;
}
function crawlPage(url, callBack = null) {
var pathArray = url.split( '/' );
var protocol = pathArray[0];
var host = pathArray[2];
var baseUrl = protocol + '//' + host;
io.emit("crawl_update", "Crawling page: " + url);
requestModule(url, async function (error, response, html) {
if (!error && response.statusCode == 200) {
var $ = cheerio.load(html);
// Get text
// console.log("------- with request module -------")
// console.log($.text());
// Get HTML
// console.log($.html());
var page = await database.collection("pages").findOne({
"url": url
});
if (page == null) {
var html = $.html();
var htmlContent = HTMLParser.parse(html);
var allAnchors = htmlContent.querySelectorAll("a");
var anchors = [];
for (var a = 0; a < allAnchors.length; a++) {
var href = allAnchors[a].getAttribute("href");
var title = allAnchors[a].innerHTML;
var hasAnyChildTag = (allAnchors[a].querySelector("div") != null)
|| (allAnchors[a].querySelector("img") != null)
|| (allAnchors[a].querySelector("p") != null)
|| (allAnchors[a].querySelector("span") != null)
|| (allAnchors[a].querySelector("svg") != null)
|| (allAnchors[a].querySelector("strong") != null);
if (hasAnyChildTag) {
continue;
}
if (href != null) {
if (href == "#" || href.search("javascript:void(0)") != -1) {
continue;
}
var first4Words = href.substr(0, 4);
if (href.search(url) == -1 && first4Words != "http") {
if (href[0] == "/") {
href = baseUrl + href;
} else {
href = baseUrl + "/" + href;
}
}
anchors.push({
"href": href,
"text": title
});
}
}
io.emit("crawl_update", htmlspecialchars("<a>") + " tags has been crawled");
var titles = await getTagContent("title", htmlContent, url);
var title = titles.length > 0 ? titles[0] : "";
io.emit("crawl_update", htmlspecialchars("<title>") + " tag has been crawled");
var h1s = await getTagContent("h1", htmlContent, url);
io.emit("crawl_update", htmlspecialchars("<h1>") + " tags has been crawled");
var h2s = await getTagContent("h2", htmlContent, url);
io.emit("crawl_update", htmlspecialchars("<h2>") + " tags has been crawled");
var h3s = await getTagContent("h3", htmlContent, url);
io.emit("crawl_update", htmlspecialchars("<h3>") + " tags has been crawled");
var h4s = await getTagContent("h4", htmlContent, url);
io.emit("crawl_update", htmlspecialchars("<h4>") + " tags has been crawled");
var h5s = await getTagContent("h5", htmlContent, url);
io.emit("crawl_update", htmlspecialchars("<h5>") + " tags has been crawled");
var h6s = await getTagContent("h6", htmlContent, url);
io.emit("crawl_update", htmlspecialchars("<h6>") + " tags has been crawled");
var ps = await getTagContent("p", htmlContent, url);
io.emit("crawl_update", htmlspecialchars("<p>") + " tags has been crawled");
var object = {
"url": url,
"anchors": anchors,
"title": title,
"h1s": h1s,
"h2s": h2s,
"h3s": h3s,
"h4s": h4s,
"h5s": h5s,
"h6s": h6s,
"ps": ps,
"time": new Date().getTime()
};
try {
await database.collection("pages").insertOne(object);
} catch (e) {
console.log(e);
}
io.emit("page_crawled", object);
io.emit("crawl_update", "Page crawled.");
} else {
io.emit("crawl_update", "Page already crawled.");
}
if (callBack != null) {
callBack();
}
}
});
}
If you refresh the page now and enter the URL of any web page and hit enter, you will see its content is stored in the Mongo DB database named web_crawler. To check the data from Mongo DB, you can download a software named Mongo DB Compass.
Show data in DataTable
Now whenever a new web page is crawled, we will display that in a table. We will be using a library called DataTable. We will also include the socket IO library for real-time communication. So include those files in your index.ejs:
Then we will create a row with 2 columns. On the left column, we will create a table to display all crawled tables. And on the right column, we will display all crawled updates e.g. “headings has been crawled”, “paragraphs” has been crawled” etc.
Then we need to initialize the data table library. Also, attach event listeners for crawl updates. Crawl updates will ab prepended in the <ul> list. The complete crawled web pages will be appended in the data table.
var table = null;
var socketIO = io("http://localhost:3000/");
var months = ["January", "February", "March", "April", "May", "June", "July", "August", "September", "October", "November", "December"];
window.addEventListener("load", function () {
table = $('#my-table').DataTable({
"order": [[ 2, "asc" ]]
});
});
socketIO.on("crawl_update", function (data) {
// console.log(data);
var html = "";
html += `<li class="list-group-item">` + data + `</li>`;
document.getElementById("my-updates").innerHTML = html + document.getElementById("my-updates").innerHTML;
document.getElementById('my-updates').scrollTop = 0;
});
socketIO.on("page_crawled", function (data) {
// console.log(data);
var date = new Date(data.time);
var time = date.getDate() + " " + months[date.getMonth() + 1] + ", " + date.getFullYear() + " - " + date.getHours() + ":" + date.getMinutes() + ":" + date.getSeconds();
table.row.add( [
"<a href='/page/" + encodeURIComponent(data.url) + "'>" + data.url + "</a>",
data.title,
time
] ).draw( false );
});
Now you will see the data in the table when you crawl some page. You can crawl as many pages as you want.
Fetch data from Mongo DB
At this point, data in the data table is only displayed when you crawl some page. But when you reload the page, the data table will be empty. However, the data is still stored in the database. Our web crawler has saved all the crawled pages in a Mongo DB collection named “pages”. So we need to populate the previously saved pages from the database in the data table when the page loads.
First, change our “/” GET route in the server.js to the following:
app.get("/", async function (request, result) {
var months = ["January", "February", "March", "April", "May", "June", "July", "August", "September", "October", "November", "December"];
var pages = await database.collection("pages").find({})
.sort({
"time": -1
}).toArray();
for (var index in pages) {
var date = new Date(pages[index].time);
var time = date.getDate() + " " + months[date.getMonth() + 1] + ", " + date.getFullYear() + " - " + date.getHours() + ":" + date.getMinutes() + ":" + date.getSeconds();
pages[index].time = time;
}
result.render("index", {
"pages": pages
});
});
And in our index.ejs inside the <tbody> tag, we will display all the pages.
If you refresh the page now, you will see all pages in the data table. You will only see the URL, title, and the time when the page was recently crawled. But we need to know the anchor tags on that page, all the headings, and paragraphs in it.
Show page content
Click on any of the links from the data table and it will take you to an error page. We need to convert that error page into a detailed page. Create a GET route in our server.js that will fetch the page from the database and send it to an HTML file.
app.get("/page/:url", async function (request, result) {
var url = request.params.url;
var page = await database.collection("pages").findOne({
"url": url
});
if (page == null) {
result.render("404", {
"message": "This page has not been crawled"
});
return false;
}
result.render("page", {
"page": page
});
});
In your views folder, create a file named 404.ejs that will be displayed when the URL is not been crawled yet.
Along with all the data on the web page, it will also show 2 buttons to “delete” and to “reindex”. Delete simply means to delete the page from the database. “Reindex” means to re-crawl the web page to fetch updated content. First, we will create a POST route for deleting the page in our server.js file.
app.post("/delete-page", async function (request, result) {
var url = request.fields.url;
await database.collection("pages").deleteOne({
"url": url
});
io.emit("page_deleted", url);
var backURL = request.header('Referer') || '/';
result.redirect(backURL);
});
And in our index.ejs we will attach an event listener that will be called when the page is deleted. In that function, we will simply remove that row from the data table.
Remove specific row from DataTable.js
socketIO.on("page_deleted", function (url) {
table
.rows( function ( idx, data, node ) {
return data[0].includes(url);
} )
.remove()
.draw();
});
This will search for the first row with the URL in its content and remove it. After removal, it will re-render the data table to reload the table.
Re-index the page
Now we need to add a function to re-index the page, which means to get the updated content of the page. As we did for delete, we will also create a form for re-indexing.
<div class="col-md-1">
<form method="POST" action="/reindex" onsubmit="return confirm('Are you sure you want to re-index this page ?');">
<input type="hidden" name="url" value="<%= page.url %>" required />
<input type="submit" class="btn btn-primary" value="Re-index" />
</form>
</div>
This will show a “Re-index” button along with a delete button. Then we need to create a POST route in our server.js:
app.post("/reindex", async function (request, result) {
var url = request.fields.url;
await database.collection("pages").deleteOne({
"url": url
});
io.emit("page_deleted", url);
crawlPage(url, function () {
var backURL = request.header('Referer') || '/';
result.redirect(backURL);
});
});
That’s it. If you run the code now, you will be able to re-index the page and update your database. You can try this feature by first crawling a URL today. Then try to crawl the same URL again a few days or weeks later.
So that’s how you can create a simple web crawler in Node JS and Mongo DB. You can check our financial ledger tutorial to learn how to create a financial ledger in Node JS and Mongo DB.
This tutorial will discuss how you can create a JWT authentication system (Login, Registration, and Logout) using JSON Web Token in Node JS and Mongo DB. You can follow the video below if you prefer a video tutorial:
First, make sure you have downloaded and installed Node JS and Mongo DB. You can also download Mongo DB Compass to view the database.
Setup the Project
First, create an empty folder anywhere on your computer and open the command prompt in it. Once CMD is open inside the folder, run the following command:
express: A framework used for better routing, middleware etc.
express-formidable: To get the values from AJAX.
mongodb: To interact with database.
http: To handle HTTP requests.
bcrypt: To encrypt and decrypt the password.
jsonwebtoken: To generate JSON Web Token used for authentication.
ejs: View engine to display HTML pages.
Now, create a file named “server.js” at the root of your folder. To start the server you can use the following command:
node server.js
But every time you made some changes in the server.js file, you have to run that command again. To prevent this, you can install the nodemon module using the following command:
npm install -g nodemon
Then you can use the following command to keep the server running:
nodemon server.js
Start the Server
Open your server.js file and write the following code in it:
// include express module
var express = require("express");
// create instance of express
var app = express();
// use express-formidable middleware
var formidable = require("express-formidable");
app.use(formidable());
// include mongodb module
var mongodb = require("mongodb");
// get mongodb client
var mongoClient = mongodb.MongoClient;
// get ObjectId, it is unique for each document
var ObjectId = mongodb.ObjectId;
// create http server from express instance
var http = require("http").createServer(app);
// include bcrypt module
var bcrypt = require("bcrypt");
// include jsonwebtoken module
var jwt = require("jsonwebtoken");
// random secret string
var accessTokenSecret = "myAccessTokenSecret1234567890";
// use public folder for css and js files
app.use(express.static(__dirname + "public"));
// use ejs engine to render html files
app.set("view engine", "ejs");
var mainURL = "http://localhost:3000";
// start the server at port 3000
http.listen(3000, function () {
console.log("Server started.");
// connect with mongodb
mongoClient.connect("mongodb://localhost:27017", function (error, client) {
// database name will be "jwt_authentication"
var database = client.db("jwt_authentication");
console.log("Database connected");
});
});
This will include all the necessary modules and start the server at port 3000. The module used for JWT authentication is named jsonwebtoken. You can access it from your browser with the following link:
http://localhost:3000/
You will see an error because there isn’t any route for the home page. Open your CMD and you will see 2 messages:
“Server started.”
“Database connected”
If you open the Mongo DB Compass you will not see a database. Because database will be created only there is at least one collection in it. So our database will be created when there is at least one user in our database.
User Registration
Go to the following address in your browser:
http://localhost:3000/signup
First, we are going to create a route in our server.js right after the database is connected.
server.js
// route for signup requests
app.route("/signup")
// get request accessed from browser
.get(function (request, result) {
// render signup.ejs file inside "views" folder
result.render("signup");
});
Now we need to create a folder named views at our root folder. Inside this folder, create a file named signup.ejs (note the extension). Also create 2 more files, header.ejs and footer.ejs.
header.ejs
<!-- menu items will be created using Javascript -->
<ul id="main-menu">
</ul>
<script>
// variables used in all files
var mainURL = "http://localhost:3000";
var accessTokenKey = "accessToken";
var user = null;
</script>
<script>
function doRegister(form) {
// disable the submit button and show "Loading..." text
form.submit.setAttribute("disabled", "disabled");
form.submit.value = "Loading...";
// create AJAX object
var ajax = new XMLHttpRequest();
// 1. method is POST
// 2. path where request will be sent
// 3. request will be asynchronous
ajax.open("POST", mainURL + "/signup", true);
// called everytime status of request changes
ajax.onreadystatechange = function () {
// when response is received from server
if (this.readyState == 4) {
// if the request is OK
if (this.status == 200) {
// response received from server
console.log(this.responseText);
// enable the submit button
form.submit.removeAttribute("disabled");
form.submit.value = "Register";
// convert the JSON string into Javascript object
var response = JSON.parse(this.responseText);
// display message
alert(response.message);
// if the user is created, then redirect to login
if (response.status == "success") {
window.location.href = "/login";
}
}
// if there is an internal server error
if (this.status == 500) {
console.log(this.responseText);
}
}
};
// create form data object from form
var formData = new FormData(form);
// actually sending the AJAX request
ajax.send(formData);
// prevent the form from submitting
return false;
}
</script>
This will show a form with input fields for name, email and password. Now we need to create a POST route in our server.js. We will chain the POST route after our “/signup” GET route.
// route for signup requests
app.route("/signup")
// get request accessed from browser
.get(function (request, result) {
// render signup.ejs file inside "views" folder
result.render("signup");
})
// post request called from AJAX
.post(async function (request, result) {
// get values from signup form
var name = request.fields.name;
var email = request.fields.email;
var password = request.fields.password;
// check if email already exists
var user = await database.collection("users").findOne({
"email": email
});
if (user != null) {
result.json({
"status": "error",
"message": "Email already exists."
});
return true;
}
// encrypt the password
bcrypt.hash(password, 10, async function (error, hash) {
// insert in database
await database.collection("users").insertOne({
"name": name,
"email": email,
"password": hash,
"accessToken": ""
});
// send the response back to client
result.json({
"status": "success",
"message": "Signed up successfully. You can login now."
});
});
});
If you run the code now, you will see a registration form. You can enter your name, email, and password and hit enter. The first time, you will see a success message and you will be redirected to the login route. You can refresh your Mongo DB Compass, and you will see your database and one collection of “users”. Inside this collection, you will see your document inserted. If you enter the same email again, you will get an error.
User Login – JWT authentication
Go to the following address in your browser:
http://localhost:3000/login
First, we are going to create a route in our server.js right after the signup route.
server.js
// route for login requests
app.route("/login")
// get request accessed from browser
.get(function (request, result) {
// render login.ejs file inside "views" folder
result.render("login");
});
Then create a file named login.ejs and write the following code in it:
<script>
function doLogin(form) {
// disable the submit button and show "Loading..." text
form.submit.setAttribute("disabled", "disabled");
form.submit.value = "Loading...";
// create AJAX object
var ajax = new XMLHttpRequest();
// 1. method is POST
// 2. path where request will be sent
// 3. request will be asynchronous
ajax.open("POST", mainURL + "/login", true);
// called everytime status of request changes
ajax.onreadystatechange = function () {
// when response is received from server
if (this.readyState == 4) {
// if the request is OK
if (this.status == 200) {
// response received from server
console.log(this.responseText);
// enable the submit button
form.submit.removeAttribute("disabled");
form.submit.value = "Login";
// convert the JSON string into Javascript object
var response = JSON.parse(this.responseText);
// if user is logged in successfully
if (response.status == "success") {
// get access token from server
var accessToken = response.accessToken;
// save in local storage
localStorage.setItem("accessToken", accessToken);
// redirect to home page
window.location.href = "/";
} else {
// display message
alert(response.message);
}
}
// if there is an internal server error
if (this.status == 500) {
console.log(this.responseText);
}
}
};
// create form data object from form
var formData = new FormData(form);
// actually sending the AJAX request
ajax.send(formData);
// prevent the form from submitting
return false;
}
</script>
This will show a form with input fields for email and password. Now we need to create a POST route in our server.js. We will chain the POST route after our “/login” GET route.
// route for login requests
app.route("/login")
// get request accessed from browser
.get(function (request, result) {
// render login.ejs file inside "views" folder
result.render("login");
})
// post request called from AJAX
.post(async function (request, result) {
// get values from login form
var email = request.fields.email;
var password = request.fields.password;
// check if email exists
var user = await database.collection("users").findOne({
"email": email
});
if (user == null) {
result.json({
"status": "error",
"message": "Email does not exists."
});
return false;
}
// check if password is correct
bcrypt.compare(password, user.password, async function (error, isVerify) {
if (isVerify) {
// generate JWT of user
var accessToken = jwt.sign({
"email": email
}, accessTokenSecret);
// update JWT of user in database
await database.collection("users").findOneAndUpdate({
"email": email
}, {
$set: {
"accessToken": accessToken
}
});
result.json({
"status": "success",
"message": "Login successfully.",
"accessToken": accessToken
});
return false;
}
result.json({
"status": "error",
"message": "Password is not correct."
});
});
});
If you run the code now, you will see a login form. You can enter your email and password and hit enter. If the credentials are correct, then it will save the access token in your local storage in your browser, and you will be redirected to the home page. You can refresh your Mongo DB Compass, and you will see that the accessToken field of the user has been updated. If you enter the wrong credentials, then it will not update the database.
Home Page
Now we need to create a route for our home page in server.js:
// route for home page
app.get("/", function (request, result) {
result.render("index");
});
Create a file named index.ejs inside views folder and write the following code in it:
<%- include ("header") %>
<%- include ("footer") %>
Now comes the footer part. Following will be the code of your footer.ejs file:
footer.ejs
<script>
// get user on page load
window.addEventListener("load", function () {
getUser();
});
function getUser() {
// check if user is logged in
if (localStorage.getItem(accessTokenKey)) {
// call AJAX to get user data
var ajax = new XMLHttpRequest();
ajax.open("POST", mainURL + "/getUser", true);
ajax.onreadystatechange = function () {
if (this.readyState == 4) {
if (this.status == 200) {
// console.log(this.responseText);
var response = JSON.parse(this.responseText);
if (response.status == "success") {
// user is logged in
window.user = response.user;
} else {
// user is logged out
localStorage.removeItem(accessTokenKey);
}
showMainMenu();
}
if (this.status == 500) {
console.log(this.responseText);
}
}
};
var formData = new FormData();
formData.append("accessToken", localStorage.getItem(accessTokenKey));
ajax.send(formData);
return false;
}
showMainMenu();
}
function doLogout() {
// send beacon to server before redirecting
var formData = new FormData();
formData.append("accessToken", localStorage.getItem(accessTokenKey));
navigator.sendBeacon(mainURL + "/logout", formData);
// remove access token from local storage
localStorage.removeItem(accessTokenKey);
return true;
}
function showMainMenu() {
var html = "";
// if user is logged in
if (localStorage.getItem(accessTokenKey)) {
html += `<li>
<a href='/login' onclick='return doLogout();'>Logout</a>
</li>`;
} else {
html += `<li>
<a href='/login'>Login</a>
</li>`;
html += `<li>
<a href='/signup'>Signup</a>
</li>`;
}
// show in main menu
document.getElementById("main-menu").innerHTML = html;
}
</script>
This will show login and signup links if the user is not logged in. If the user is logged in, then it will display a logout button. When the logout button is clicked, it will be redirected to the “/login” route. But before redirecting, it will send a beacon to the server to empty the accessToken field in the database. So we need to create POST routes for “/getUser” and for “/logout”:
// return user data using access token
app.post("/getUser", async function (request, result) {
var accessToken = request.fields.accessToken;
var user = await database.collection("users").findOne({
"accessToken": accessToken
});
if (user == null) {
result.json({
"status": "error",
"message": "User has been logged out. Please login again."
});
return false;
}
result.json({
"status": "success",
"message": "Data has been fetched.",
"user": user
});
});
// empty the access token of user (logout)
app.post("/logout", async function (request, result) {
var accessToken = request.fields.accessToken;
await database.collection("users").findOneAndUpdate({
"accessToken": accessToken
}, {
$set: {
"accessToken": ""
}
});
result.json({
"status": "success",
"message": "User has been logged out."
});
});
With that, your authentication is completed. You can log in and log out now. Once you log out and refresh the Mongo DB Compass, you will see the accessToken field becomes empty. And it will again have the value once the user logged in again.
Why we used Beacon ?
You might have noticed that we have used Beacon instead of AJAX when the logout button is clicked. The reason for this is when the browser navigates from one page to another, the AJAX calls are sometimes aborted by the browser. To prevent this, we use beacons.
Get Data using AJAX
Now that your authentication system is done, you might want to show data to logged-in users only. So for example, you have a blogging website and you want to show your blog posts to logged-in users only. So you need to do the following steps.
1) Create Variable and a Function
In the file where you want to fetch and display the data, create the following variable and a function in Javascript.
<script>
var onPostsPage = true;
function getPosts() {
// call ajax
}
</script>
If you want to learn how to call an AJAX request and show data from it, follow this tutorial.
2) Call Function if Variable is Set
In your footer.ejs, when the user is logged-in, check if this variable is set and is true. If yes, then simply call that function.
// user is logged in
window.user = response.user;
if (typeof onPostsPage !== "undefined" && onPostsPage) {
getPosts();
}
So, that’s how you can create a JWT authentication system In Node JS and Mongo DB.
When you hear the word “social networking site”, the first name that came to your mind must be Facebook. Then Instagram, Twitter, etc. We create a social networking site project just to give you an idea of how things work. It is not the exact code used in tech giants like Facebook. But it gives you an idea of how they do it.
Track system mails: All the emails sent from the system (Mail::to) function will be stored in a database. And the admin can view all the emails from the admin panel.
Online/offline status: Now you can know when your friends are online and when they are offline. This will be helpful because you can just leave a message if the person is currently offline.
Last scene of the user: You can also see your friend’s last scene active. This will help in situations like earthquakes etc. you can know which friends are not active in recent times.
Track user activity: Admin can view the activities of users like when they are logged in when they created a page when they joined a group etc.
