The final project for my Data Engineer training at DataScientest, Travel Planner, heavily relied on the library Leaflet: an open-source JavaScript library for mobile friendly interactive maps. For this python project we decided to use the python wrappers for leaflet, namely dash-leaflet and folium, depending on the tasks at hand and the development frameworks (dash/streamlit).
Many times I wished I had the time, just a couple of days, to get up to speed on javascript and Node.js environment to use the full potential of leaflet and produce a better interface for the project. I tested a couple of things but I was not ready to have my system messed up with improper Node.js installs.
A couple of days ago, I inadvertently ended up here and found out that Node.js, among other frameworks, could be managed with my favourite python environment manager Conda!
Indeed, setting up a ready to use Node.js environment amounts to
# create a NodeJS virtual environment
conda create -n nodeenv nodejs
# activate the environment
conda activate nodeenv
Great, no excuse, I am all set for learning Node.js! Except that I still do not have time to dive into the many sources around... Why not starting from use-cases that I am familiar with in python and ask ChatGPT for its help?
ChatGPT :
Certainly! To create a web app showing a Leaflet map with two points located at Condat sur Vézère and Bordeaux (France), you can follow the steps below. Please note that you'll need to have Node.js installed on your system.
1. Set up the project:
Create a new directory for your project and navigate into it in the terminal. Initialize a new Node.js project and install the required dependencies.
mkdir leaflet-map-app
cd leaflet-map-app
npm init -y
npm install express leaflet
2. Create the server file:
Create a new file called `server.js` in the project directory and add the following code:
const express = require('express');
const app = express();
app.use(express.static('public'));
app.listen(3000, () => {
console.log('Server is running on http://localhost:3000');
});
3. Create the HTML file:
Create a new directory called `public` in the project directory. Inside the `public` directory, create a new file called `index.html` and add the following code:
<!DOCTYPE html>
<html>
<head>
<title>Leaflet Map</title>
<link rel="stylesheet" href="https://unpkg.com/leaflet/dist/leaflet.css" />
<style>
#map {
height: 400px;
}
</style>
</head>
<body>
<div id="map"></div>
<script src="https://unpkg.com/leaflet/dist/leaflet.js"></script>
<script>
var map = L.map('map').setView([45.144, 0.987], 6);
L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', {
attribution: 'Map data © <a href="https://www.openstreetmap.org/">OpenStreetMap</a> contributors'
}).addTo(map);
var condatMarker = L.marker([45.148, 1.183]).addTo(map);
condatMarker.bindPopup("<b>Condat sur Vézère</b>").openPopup();
var bordeauxMarker = L.marker([44.837, -0.579]).addTo(map);
bordeauxMarker.bindPopup("<b>Bordeaux</b>").openPopup();
</script>
</body>
</html>
4. Run the application:
In the terminal, navigate to the project directory and start the server:
node server.js
Now, you can access the web app by opening your browser and visiting `http://localhost:3000`. You should see a Leaflet map with markers at Condat sur Vézère and Bordeaux in France.
Feel free to customize the map's initial view and markers' positions according to your needs.
Ok, that was quick, easy and fully functional, let's add some features:
- some tourist locations,
- some graph oriented database to host these locations,
- and finally let's wrap the whole project as a docker compose application.
ChatGPT :
Certainly! To add markers for the top tourist attractions between Bordeaux and Condat sur Vézère, along with distinct colors and popups containing short descriptions, you can modify the JavaScript code in the `index.html` file. Here's an updated version of the code:
<!DOCTYPE html>
<html>
<head>
<title>Leaflet Map</title>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.7.1/leaflet.css" />
<style>
#map {
height: 400px;
}
</style>
</head>
<body>
<div id="map"></div>
<script src="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.7.1/leaflet.js"></script>
<script>
const map = L.map('map').setView([45.1575, 1.1983], 7);
L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', {
attribution: 'Map data © <a href="https://openstreetmap.org">OpenStreetMap</a> contributors'
}).addTo(map);
const attractions = [
{
name: 'La Cité du Vin',
description: 'La Cité du Vin is a cultural facility in Bordeaux, France that is dedicated to wine. It offers various exhibits and workshops to explore the history and culture of wine.',
coordinates: [44.8563, -0.5576],
color: 'red'
},
{
name: 'Dune of Pilat',
description: 'The Dune of Pilat is the tallest sand dune in Europe, located near Arcachon Bay in western France. It offers stunning views of the surrounding area and attracts numerous visitors.',
coordinates: [44.5796, -1.2170],
color: 'blue'
},
{
name: 'Lascaux Cave',
description: 'Lascaux Cave is a complex of caves near Montignac, France, famous for its Paleolithic cave paintings. The paintings, dating back to the Upper Paleolithic period, are well-preserved and depict various animals.',
coordinates: [45.0585, 1.1796],
color: 'green'
},
{
name: 'Sarlat-la-Canéda',
description: 'Sarlat-la-Canéda is a picturesque medieval town in the Dordogne department of southwestern France. It is known for its well-preserved historic buildings, narrow streets, and vibrant market.',
coordinates: [44.8906, 1.2183],
color: 'orange'
},
{
name: 'Les Jardins de Marqueyssac',
description: 'Les Jardins de Marqueyssac is a stunning 17th-century garden located near Vézac, France. It features beautifully manicured hedges, scenic walking paths, and panoramic views of the Dordogne Valley.',
coordinates: [44.8408, 1.1667],
color: 'purple'
}
];
attractions.forEach(attraction => {
const marker = L.marker(attraction.coordinates, { icon: L.icon({ iconUrl: `https://raw.githubusercontent.com/pointhi/leaflet-color-markers/master/img/marker-icon-${attraction.color}.png` }) })
.addTo(map)
.bindPopup(`<b>${attraction.name}</b><br>${attraction.description}`);
});
</script>
</body>
</html>
In the updated code, I've added an `attractions` array that contains information about the top tourist attractions. Each attraction has a name, a description, coordinates (latitude and longitude), and a color for the marker icon. The code then iterates over the `attractions` array, creates a marker for each attraction with the specified color icon, and binds a popup containing the attraction's name and description
Note that the code uses custom marker icons from the Leaflet Color Markers project on GitHub. The URL of each marker icon is generated dynamically based on the specified color in the `attractions` array.
When you run the updated code, you'll see the Leaflet map with markers for the tourist attractions between Bordeaux and Condat sur Vézère. Each marker will have a distinct color, and clicking on a marker will display a popup with a short description of the attraction.
As asked, ChatGPT added tourist attractions with some clear and concise description with distinct markers and the code is functional. Note however that Bordeaux and Condat-sur-Vézère have been removed, we will save our complaints for later, the remaining request will certainly be much harder to get right.
ChatGPT :
Certainly! Here's a step-by-step guide on using a graph-oriented database to store and fetch tourist attraction data, along with modifications to the server.js and index.html files to retrieve the data and display it on the map.
1. CSV File containing the Attractions
Create a CSV file named `attractions.csv` with the following columns: `name`, `latitude`, `longitude`, and `description`. Populate the file with the tourist attraction data, for example:
name,latitude,longitude,description
La Cité du Vin,44.8563,-0.5576,La Cité du Vin is a cultural facility in Bordeaux, France that is dedicated to wine.
Dune of Pilat,44.5796,-1.2170,The Dune of Pilat is the tallest sand dune in Europe, located near Arcachon Bay in western France.
Lascaux Cave,45.0585,1.1796,Lascaux Cave is a complex of caves near Montignac, France, famous for its Paleolithic cave paintings.
Sarlat-la-Canéda,44.8906,1.2183,Sarlat-la-Canéda is a picturesque medieval town in the Dordogne department of southwestern France.
Les Jardins de Marqueyssac,44.8408,1.1667,Les Jardins de Marqueyssac is a stunning 17th-century garden located near Vézac, France.
Save this `attractions.csv` file in the project directory.
2. Graph-Oriented Database Installation
For a graph-oriented database, we'll use Neo4j. Here are the installation instructions for Ubuntu 22.04:
Step 1: Import the Neo4j GPG key:
wget -O - https://debian.neo4j.com/neotechnology.gpg.key | sudo apt-key add -
Step 2: Add the Neo4j repository to APT sources:
echo 'deb https://debian.neo4j.com stable 4.3' | sudo tee -a /etc/apt/sources.list.d/neo4j.list
Step 3: Update the package lists:
sudo apt update
Step 4: Install Neo4j:
sudo apt install neo4j
Step 5: Start the Neo4j service:
sudo service neo4j start
3. Inserting CSV File into Neo4j
We'll use the Neo4j import tool to insert the CSV file into the database. Run the following command in the terminal:
neo4j-admin import --nodes=Attraction=attractions.csv
This command imports the attractions.csv file as nodes of type "Attraction" in the Neo4j database.
4. Modifying server.js and index.html
Now, let's modify the server.js and index.html files to fetch the attractions data from the Neo4j database and display them on the map.
Modify the server.js file as follows:
const express = require('express');
const neo4j = require('neo4j-driver');
const app = express();
app.use(express.static('public'));
const driver = neo4j.driver('bolt://localhost:7687', neo4j.auth.basic('neo4j', 'password'));
app.get('/attractions', async (req, res) => {
const session = driver.session();
try {
const result = await session.run('MATCH (a:Attraction) RETURN a.name AS name, a.latitude AS latitude, a.longitude AS longitude, a.description AS description');
const attractions = result.records.map(record => record.toObject());
res.json(attractions);
} catch (error) {
console.error('Error retrieving attractions:', error);
res.sendStatus(500);
} finally {
session.close();
}
});
app.listen(3000, () => {
console.log('Server is running on http://localhost:3000');
});
In the modified server.js file, we've added the `neo4j-driver` package to establish a connection with the Neo4j database. The /attractions route retrieves the attractions data from the database and sends it as JSON in the response.
Next, modify the index.html file as follows:
<!DOCTYPE html>
<html>
<head>
<title>Leaflet Map</title>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.7.1/leaflet.css" />
<style>
#map {
height: 400px;
}
</style>
</head>
<body>
<div id="map"></div>
<script src="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.7.1/leaflet.js"></script>
<script>
const map = L.map('map').setView([45.1575, 1.1983], 7);
L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', {
attribution: 'Map data © <a href="https://openstreetmap.org">OpenStreetMap</a> contributors'
}).addTo(map);
fetch('/attractions')
.then(response => response.json())
.then(attractions => {
attractions.forEach(attraction => {
const marker = L.marker([attraction.latitude, attraction.longitude], { icon: L.icon({ iconUrl: `https://raw.githubusercontent.com/pointhi/leaflet-color-markers/master/img/marker-icon-${attraction.color}.png` }) })
.addTo(map)
.bindPopup(`<b>${attraction.name}</b><br>${attraction.description}`);
});
})
.catch(error => console.error('Error retrieving attractions:', error));
</script>
</body>
</html>
In the modified index.html file, we've replaced the hardcoded attractions array with a fetch request to the `/attractions` route of our server. The response containing the attractions data is then used to create markers on the map, similar to the previous code.
Note: Update the `neo4j.auth.basic('neo4j', 'password'`) line in `server.js` with your Neo4j username and password.
5. Run the Web App
Ensure that Neo4j is running by executing the command `sudo service neo4j status`. Then, start the Node.js server by running `node server.js` in the project directory. Open your web browser and visit `http://localhost:3000` to see the Leaflet map with markers representing the attractions fetched from the Neo4j database.
That's it! You have now modified the server.js and index.html files to fetch attractions data from a graph-oriented database (Neo4j) and display them on the Leaflet map.
That is a rather long answer with some good parts and some bad parts:
- In the csv file, the descriptions contain commas therefore choosing commas as separator is not appropriate. When asked to modify this, ChatGPT chose to keep the comma for separator and replaced the commas in the text by '|' which does not make much sense in this context.
- There isn't any script to import the CSV file into neo4j.
- The html reads the attractions from the CSV instead of querying the database.
- The javascript that displays the markers refers to a field `color` which is never defined.
- The install process for neo4j went smoothly but I couldn't get the command `neo4j-admin import --nodes=Attraction=attractions.csv` to work. `Neo4j-driver` had to be added to the `package.json` file.
ChatGPT :
Certainly! Here's an example of how you can use Docker Compose to wrap up the project into a containerized application.
Create the following files in your project directory:
const express = require('express');
const app = express();
const neo4j = require('neo4j-driver');
const driver = neo4j.driver('bolt://neo4j:7687', neo4j.auth.basic('neo4j', 'password'));
app.use(express.static('public'));
app.get('/attractions', async (req, res) => {
const session = driver.session();
const query = 'MATCH (a:Attraction) RETURN a.name AS name, a.latitude AS latitude, a.longitude AS longitude, a.description AS description';
try {
const result = await session.run(query);
const attractions = result.records.map(record => record.toObject());
res.json(attractions);
} catch (error) {
console.error('Error retrieving attractions:', error);
res.status(500).send('Error retrieving attractions');
} finally {
await session.close();
}
});
const port = 3000;
app.listen(port, () => {
console.log(`Server is running on http://localhost:${port}`);
});
<!DOCTYPE html>
<html>
<head>
<title>Leaflet Map</title>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.7.1/leaflet.css" />
<style>
#map {
height: 400px;
z-index: 0;
}
</style>
</head>
<body>
<div id="map"></div>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.6.0/jquery.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.7.1/leaflet.js"></script>
<script>
var map = L.map('map').setView([45.188529, 0.874760], 8);
L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', {
attribution: 'Map data © <a href="https://www.openstreetmap.org/">OpenStreetMap</a> contributors',
maxZoom: 18,
}).addTo(map);
$.getJSON('/attractions', function (data) {
console.log('Attractions data imported successfully!', data);
data.forEach(function (attraction) {
var marker = L.marker([attraction.latitude, attraction.longitude]).addTo(map);
marker.bindPopup(attraction.description);
});
});
</script>
</body>
</html>
# Dockerfile
# Base image
FROM node:14
# Set working directory
WORKDIR /app
# Copy package.json and package-lock.json
COPY package*.json ./
# Install dependencies
RUN npm install
# Copy server.js and index.html
COPY server.js .
COPY index.html .
# Expose port
EXPOSE 3000
# Start the server
CMD ["node", "server.js"]
version: '3'
services:
web:
build:
context: .
dockerfile: Dockerfile
ports:
- 3000:3000
volumes:
- ./attractions:/app/attractions
neo4j:
image: neo4j:latest
container_name: neo4j
environment:
- NEO4J_AUTH=neo4j/password
- NEO4J_ACCEPT_LICENSE_AGREEMENT=yes
ports:
- 7474:7474
- 7687:7687
volumes:
- ./import:/var/lib/neo4j/import
In the `docker-compose.yml` file, we define two services: neo4j for the Neo4j database and web for the Node.js web server. The services are linked, and the necessary ports are exposed.
Make sure the attractions.csv file is present in the `import` directory.
To build and run the Docker containers, execute the following command in the project directory:
docker-compose up -d
This will build the Docker images and start the containers. Access the Neo4j browser by opening the URL on `localhost:7474` in your web browser. Login with the credentials `neo4j/password` as specified in the `docker-compose.yml` file.
In the Neo4j browser, run the following Cypher query to create the Attraction nodes and import the data from the attractions.csv file:
LOAD CSV WITH HEADERS FROM 'file:///attractions.csv' AS row FIELDTERMINATOR ';'
CREATE (:Attraction {
name: row.name,
latitude: toFloat(row.latitude),
longitude: toFloat(row.longitude),
description: row.description
});
This query reads the attractions.csv file and creates Attraction nodes in the database based on the file's contents.
The web application is accessible at `http://localhost:3000`, it displays the Leaflet map with markers for the attractions imported into Neo4j.
By using Docker Compose, both the Node.js server and the Neo4j database will be running in separate containers with their dependencies properly configured.
I hope this helps! If you have any further questions, please let me know.
The final project can be found in the github repository gtp-prompts/nodejs-neo4j.