Backend Developer @ lobstr.io Python Scraping Celery

Backend Developer @ lobstr.io (Python Scraping, Celery)

About lobstr.io

lobstr.io is a no-code scraping SaaS used by 25,000+ active users monthly. We empower non-technical users to extract and automate structured data from complex websites.

We're looking for a full-time Backend Developer to join our core team and help us build, maintain, and scale our scraping infrastructure.

Your Mission

• Build and maintain Python scrapers
• Maintain and scale our asynchronous job pipeline (Celery + Redis)
• Improve internal tools and admin panels
• Solve scraping challenges: detection evasion, speed, modularity

Our Stack

• Python, Django, PostgreSQL
• Celery / Redis
• GitLab, Notion, Trello, Slack
• Remote Linux servers, VSCode, Sentry

Why Join Us

• 150% YoY growth
• Public scraper roadmap (goal: +4/month)
• Profit-sharing on monthly MRR growth
• Real impact on architecture and product
• Full remote with paid time off (1 day/month)

Our Work Culture

• 1 short dev sync per week
• 1 monthly all-hands
• You manage your schedule - we care about delivery, not hours

Trial & Salary

• 3-month trial: $2,000/month
• Then: $2,500/month + MRR growth sharing (20%)
• Contract: freelance / contractor via Deel
• Possible long-term contract or CTO-track if the fit is right

Qualifications:

• Proven experience building and maintaining Python web applications
• Strong understanding of asynchronous programming concepts (Celery, Redis)
• Experience with web scraping techniques and challenges
• Excellent problem-solving and debugging skills
• Ability to work independently and as part of a remote team

Qualifications

Required:

• Proven experience: 2+ years of experience developing backend systems using Python.
• Django expertise: Strong understanding and practical experience with the Django framework.
• Database proficiency: Experience working with PostgreSQL databases, including querying and data manipulation.
• Asynchronous processing: Familiarity with Celery and Redis for building and managing asynchronous job queues.
• Git and collaboration: Experience using Git for version control and collaborating on codebases.

Preferred:

• Web scraping experience: Knowledge of web scraping techniques and best practices, including handling dynamic content and anti-scraping measures.
• API development: Experience building and consuming REST APIs.
• Linux server administration: Basic understanding of Linux server environments and command-line tools.
• Problem-solving mindset: Strong analytical and problem-solving skills with a passion for finding efficient solutions.
• Communication skills: Excellent written and verbal communication skills for collaborating effectively within a remote team.

Key Responsibilities

Scraping Infrastructure:

• Design, develop, and maintain Python-based web scrapers for diverse websites.
• Implement robust error handling and exception management strategies.
• Optimize scraper performance for speed and efficiency.
• Develop and maintain tools for detecting and evading anti-scraping measures.

Asynchronous Job Processing:

• Manage and scale our asynchronous job pipeline using Celery and Redis.
• Monitor job queue performance and implement optimizations for throughput.
• Develop and maintain robust logging and monitoring systems for the job pipeline.

Internal Tools & Development:

• Contribute to the development and maintenance of internal tools and admin panels.
• Collaborate with the product team to implement new features and functionalities.
• Write clean, well-documented, and maintainable code.
• Participate in code reviews and contribute to the overall code quality.

lobstr.io Backend Developer Selection Process

lobstr.io is seeking a talented Backend Developer to join our remote-first team. Here's how our selection process works:

1. Application Review: We'll carefully review your resume and cover letter, focusing on your Python, Django, and Celery experience, as well as your understanding of scraping techniques and asynchronous job processing.

2. Technical Assessment: You'll be presented with a coding challenge designed to assess your skills in Python, web development, and scraping. This will be a practical exercise simulating real-world tasks.

3. Interview Rounds: Successful candidates will participate in two rounds of interviews:

   • Initial Interview: A conversation with a team member to discuss your experience, motivations, and fit within our culture.
   • Technical Interview: A deeper dive into your technical skills and problem-solving abilities, focusing on your understanding of the job requirements and your approach to tackling challenges.

4. Trial Period: If you impress us, you'll enter a 3-month trial period as a contractor. This allows us to evaluate your performance and collaboration style in a real-world setting.

5. Onboarding: Upon successful completion of the trial, we'll offer you a full-time contract with a competitive salary and profit-sharing opportunities.

We're excited to meet talented developers who are passionate about building innovative solutions!

About lobstr.io

lobstr.io is a no-code web scraping SaaS empowering over 25,000 active users monthly to extract and automate structured data from complex websites. We believe everyone should have access to the power of data, regardless of their technical expertise.

Our mission is to democratize web scraping by providing a user-friendly platform that simplifies the process and unlocks valuable insights. We're a remote-first, async-friendly team with a low-meeting culture, focused on delivering impactful results. We're passionate about building innovative solutions and fostering a collaborative environment where everyone can thrive.