Requirements
- At least 5 years of experience building and operating production data platforms, particularly those involving multi-source ingestion and end-to-end platform ownership.
- Production experience with workflow orchestration tools, including managing dependencies, retries, recovery processes, and complex workflows. Airflow is preferred, while Dagster, Prefect, Temporal, or
- Hands-on production experience with dbt and Google Cloud Platform (GCP).
- Strong Python programming skills, including experience with asynchronous HTTP requests and concurrent processing.
- Advanced SQL and PostgreSQL skills, including data modeling, query optimization, and incremental loading techniques.
- Experience designing and operating event-driven processing systems, messaging queues, and reliable data pipelines.
Benefits
- Equity
Skills
Nice to have
Full job description
Accountabilities
- Design and develop a multi-source data platform supporting concurrent ingestion, event-driven processing, and orchestrated batch workflows.
- Build reliable data ingestion systems connecting to provider APIs, webhooks, files, and web scraping sources, with appropriate rate limiting and robust error-handling mechanisms.
- Implement scheduled workflows and historical data backfills using orchestration tools such as Airflow, Dagster, or Prefect.
- Develop event-driven processing systems using messaging infrastructure and task queues, including retries, dead-letter queues, and controlled replay capabilities.
- Store original source payloads in cloud storage and use dbt to transform data into structured raw, standardized, and curated layers.
- Implement idempotent upserts, data contracts, automated validation, pipeline testing, and monitoring to detect freshness issues and processing failures.
- Design, maintain, and optimize PostgreSQL schemas and data-loading strategies to support performance, reliability, and scalability.
- Integrate external provider enrichment and structured data extraction capabilities using Large Language Models (LLMs).
- Take end-to-end ownership of the platform, covering architectural decisions, implementation, deployment, ongoing maintenance, and production reliability.
- At least 5 years of experience building and operating production data platforms, particularly those involving multi-source ingestion and end-to-end platform ownership.
- Production experience with workflow orchestration tools, including managing dependencies, retries, recovery processes, and complex workflows. Airflow is preferred, while Dagster, Prefect, Temporal, or comparable technologies are also relevant.
- Hands-on production experience with dbt and Google Cloud Platform (GCP).
- Strong Python programming skills, including experience with asynchronous HTTP requests and concurrent processing.
- Advanced SQL and PostgreSQL skills, including data modeling, query optimization, and incremental loading techniques.
- Experience designing and operating event-driven processing systems, messaging queues, and reliable data pipelines.
- Knowledge of data contracts, raw and curated storage architectures, data retention policies, automated testing, and pipeline monitoring.
- Ability to design resilient ingestion and transformation processes that handle failures, maintain data consistency, and support recovery.
- Experience building data platforms or substantial pipelines from the ground up is a strong advantage.
- Familiarity with web scraping, entity resolution, company data enrichment, and LLM-based data extraction is beneficial.
- Experience working in a small engineering team or startup environment is also an advantage.
- Annual salary range of USD 100,000–140,000.
- Fully remote working arrangement within eligible European locations, including Switzerland, the United Kingdom, Norway, Ukraine, Iceland, and Liechtenstein.
- Opportunity to take end-to-end ownership of a data platform and contribute to its architecture and long-term development.
- Opportunity to work with modern data engineering technologies, cloud infrastructure, event-driven systems, and LLM-powered data enrichment.
- Visa sponsorship is not available for this position.
We appreciate your interest and wish you the best!
Why Apply Through Jobgether?
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.
Opens Jobgether’s listing on LinkedIn in a new tab.