Tuesday, 5 December, 2023 UTC
Summary
Building a niche-related job board like remote jobs, software development jobs, etc. has been all the rage recently.
In fact, nowadays you can hardly Google for jobs without being redirected to a job board first.
At first glance, it may seem incredibly complicated to create one, too. After all, you would need to either receive or find all of these jobs to then post on your job board.
But what if I told you that many of these job boards outside of the likes of Indeed and other top job boards are primarily just scraping LinkedIn jobs and then displaying the results and monetizing it in the process?
Remote oriented job boardPlus, you too could create your very own job board for your desired niche much easier than you think, which means:
- No scraping of your own required
- No finding job posts manually
- No having to sift through job post submissions
We can actually automate nearly the entire process of running a job board. In this blog post, I’ll show you how. Let’s dive in.
First things first:
Why scrape LinkedIn?
LinkedIn is the single largest B2B platform in the world. There’s no other platform that contains as much B2B data as LinkedIn.
Many people recognize this fact. But they also recognize it’s pretty complex and is a game of cat and mouse.
LinkedIn doesn’t want you to scrape their website. They also don’t even really allow you to pay for it. Their API is extremely limited, and they limit access to it to any company that competes in any relatively adjacent industry (which a job board would definitely fall under).
This means two things:
- While the B2B data on LinkedIn is incredibly valuable, many opt to skip scraping it because it’s a headache to scrape. Which is fair.
- To obtain the data on LinkedIn, it’s generally best to obtain it through a third-party API that does not limit you, rather than LinkedIn’s API.
That said: that’s what Proxycurl is – a third-party API primarily powered by LinkedIn’s data and enhanced through other sources.
We deal with all of the headaches involved with scraping LinkedIn – like proxies, CAPTCHA solving, and so on – and deliver the B2B data in a convenient and easy to use format (an API).
By the way, if you’re wondering if this is legal, we only work with publicly available data. US law and past court cases have cemented the legality of publicly scraping LinkedIn.
Why create yet another job board?
If you’re curious why everyone’s been creating a job board recently, like the answer to the reason why most businesses are created: it’s profitability.
Staffing/recruiting is a profitable business. Most businesses are constantly recruiting new talent. Especially unique individualized talent that’s not easy to find.
That said, the monetization strategy for job boards tends to vary.
Some of these sites operate a pay to post strategy for example, but for the most part the main monetization strategy of these job boards is simply data harvesting.
We Work Remotely's monetization strategy ($299 job posts)You’ll find they often have you essentially fill out an application on the job board website, and then redirect you to the actual job. Or at least collect some basic information and your email address.
This is because they can then go and sell your unique relevant information to applicable companies to recruit you, or proceed to follow up and try to gain a recruitment commission.
Alrighty, now that we’ve covered the basics, let’s dive into actually building a job board:
Who is your job board targeting?
The first step to building a job board is picking out a specific niche.
I recommend something more specific than just “remote jobs” for instance – and also just “software development” job boards do seem a bit saturated now.
How can you further differentiate your job board from your competitors and make it a job board that’s genuinely valuable and helpful for a specific target audience?
Rather than just being a “software development” job board, take it down another level. So for example: you could be a Python only data analytics job board. Obviously this is just an example, but you get the point. Pick a job board that’s unique from the broad job boards already existing.
This carries two major benefits:
- By specifying your job board, you stand out and it’s easier to attract talent rather than trying to compete with broad and established job boards
- If you make your job board known as a source of individualized talent, you can make strong relationships with relative companies that need the kind of talent coming to your job board.
Companies will line up to give you tons of money if you can find talented staff that know what they're doing and have the skills to actually move the needle.
So before proceeding with this article, take a second to think: What’s a unique angle that isn’t overly saturated and is in demand by businesses with money?
The “with money” part is often why “software development” is such a frequently targeted job board audience. SaaS companies and the type of companies that need these skilled employees are often lined with more cash than someone like an eCommerce business looking for a customer support representative, for example.
Building a job board with Proxycurl
Since this method skips actually scraping the job posting data, this is probably easier than you think.
The top down overview of building a job board is very simple:
- Have a LinkedIn job scraper pulling data from LinkedIn (in this tutorial we will use Proxycurl)
- Display the data pulled from LinkedIn on your job board
- Have a monetization strategy in place (again, this usually revolves around working with relative companies to gain recruitment commissions, etc.)
The primary API endpoint that’s going to be the most relevant here is our Job Search Endpoint – this will allow you to specify identifying job characteristics and then get the LinkedIn job posting results returned back to you.
We’ll also be using Python to assist us with this, but you can use any programming language of your choice.
Let’s get started:
Step 1) Setting up your development environment
The first thing you’ll need is Python installed on your machine (I use PyCharm as my IDE). Then you’ll also need a web framework for Python.
In this case, we’ll use Flask which you can install with the following pip command:
pip install flask
Next you’ll create a new Python file for your Flask app, for example
app.py and setup a basic Flask application:
from flask import Flask, render_template
app = Flask(__name__)
@app.route('/')
def index():
return render_template('index.html')
if __name__ == '__main__':
app.run(debug=True)
Create a folder named
templates in the same directory as app.py. Inside that folder, create an HTML file named index.html – this will serve as the main page of your job board.A simple
index.html could look like this:<!DOCTYPE html>
<html>
<head>
<title>Data Analyst Job Board</title>
</head>
<body>
<h1>Data Analyst Job Listings</h1>
<div id="jobs">
<!-- Job listings will go here -->
</div>
</body>
</html>
Now let’s integrate Proxycurl’s API with our Flask application. Keep in mind you’ll need to replace
Your_API_Key with your actual Proxycurl API key. Again, we are indeed using the Job Search Endpoint here.That said, let’s say we wanted to create a job board specifically for “data analysts” – by modifying
app.py, we could accomplish this:
import requests
from flask import Flask, render_template, jsonify
app = Flask(__name__)
@app.route('/')
def index():
return render_template('index.html')
@app.route('/api/jobs')
def get_jobs():
api_key = `Your_API_Key`
headers = {'Authorization': 'Bearer ' + api_key}
api_endpoint = 'https://nubela.co/proxycurl/api/v2/linkedin/company/job'
params = {
'keyword': 'data analyst',
}
response = requests.get(api_endpoint, params=params, headers=headers)
print("API Response:", response.json()) # This line prints the API response
if response.status_code == 200:
return jsonify(response.json())
else:
return jsonify({'error': 'Failed to fetch jobs'})
if __name__ == '__main__':
app.run(debug=True)
The key part of this script that's specifying the type of job's it's searching for is this part right here:
'keyword': 'data analyst',
There are all different ways of customizing this script, but the first thing you're going to want to modify is that part right there.
You can learn more about targeting with our Job Search Endpoint here.
Step 5) Fetch and display jobs in the frontend
We need to use some JavaScript to fetch the job data data from the Job Search Endpoint and display it on the page. This can be done using AJAX.
Go ahead and add the following script to
index.html like so:<!DOCTYPE html>
<html>
<head>
<title>Data Analyst Job Board</title>
</head>
<body>
<h1>Data Analyst Job Listings</h1>
<div id="jobs">
<!-- Job listings will be added here -->
</div>
<script>
document.addEventListener('DOMContentLoaded', function() {
fetch('/api/jobs')
.then(response => response.json())
.then(data => {
const jobsContainer = document.getElementById('jobs');
data.job.forEach(job => { // Adjusted to match the API response structure
const jobElement = document.createElement('div');
jobElement.innerHTML = `
<h2>${job.job_title}</h2>
<p>Company: ${job.company}</p>
<p>Location: ${job.location}</p>
<p>Listed on: ${job.list_date}</p>
<a href="${job.job_url}" target="_blank">View Job</a>
`;
jobsContainer.appendChild(jobElement);
});
})
.catch(error => console.error('Error:', error));
});
</script>
</body>
</html>
Now we can go ahead and run our Flask application. You can click run if your using PyCharm, or execute the following command:
python app.py
Your job board is now live! Go ahead and visit
What our (ugly) job board looks likehttp://localhost:5000 in your browser to see your job board:Definitely ugly, but you can edit the index according to your liking. The important part is that all of the functionality you need to create a job board is there.
By the way, if you go to
API response seen from our job boardStep 7) Customize to your liking
http://localhost:5000/api/jobs you can see the raw API response:You now have a way to pull recent job postings at scale. You simply need to further edit the above script to your liking, and work on bringing your job board to life (as well as adding a monetization strategy, but that usually comes later on).
There are many different options for hosting your Python website, but one of the easiest and most frequently used ones would be Heroku.
All you need to do is put your code within a GitHub repository, sync it, and specify that the code inside is in fact Python:
Running Python on a Heroku applicationThen you’re good to go and can deploy your GitHub code onto a live Heroku Python application:
Deploying Python code live on HerokuHow much does it cost to create a job board with Proxycurl?
The actual hosting of your application is one thing, but in terms of our end, the Job Search Endpoint, it costs 2 credits per successful job post returned.
You see: our API costs credits. Every endpoint will have its own respective credit usage cost.
We also have both pay-as-you-go plans which will allow you to buy 100 credits for $10 for instance, or subscription plans that will allow you to get substantially more credits for the same price.
So that means with the pay-as-you-go-plans (the most expensive), every job post you’ll pull will cost you $0.20 cents. Not bad.
Definitely leaves enough meat on the bone for you to be able to make some more on top.
Create your Proxycurl account
You've now seen the simplicity and power of building a specialized job board with Proxycurl.
With the basics in place and your unique niche identified, you're well on your way to creating a platform that connects talented professionals with the right opportunities.
That means the next step is to create your Proxycurl account, grab your API key, and get your job board started today!
Click here to create your Proxycurl account now for free.
P.S. Have any questions? No problem, we have answers. Just reach out to us at "[email protected]" and we'll get you taken care of ASAP!