Description
Senior Data Engineer
Location: United States
Type: Full Time
POSITION SUMMARY
We’re looking for a Senior Data Engineer to build and enhance our innovative monitoring application. You will be a key voice as the company evolves this exciting application. The ideal candidate is someone who has strong data architecture expertise, diverse database experience, and understands how to efficiently and effectively coordinate data preparation tasks. This person's main focus will be designing data transformation processes, developing/modifying data models, and automating common analysis workflows. This will include developing data ETL/ELT pipelines via SQL/Python, developing automated processes to consume new data and augment / increment existing data, identifying opportunities for efficiency, keeping accurate and complete project documentation, and performing quality testing and data assurance.
RESPONSIBILITIES
- Quickly bridge-the-gap between source data and models needed to analytically process the data. Ensure all efforts contemplate a multi tenant approach and the automation potential.
- Ensure integrity and security of data.
- Enhance, standardize, and join different data sets to achieve data science and analytics objectives.
- Research and build efficient and scalable data storage and retrieval systems that enable interactive reporting on high dimensional data.
- Maintain accurate, complete, and current documentation.
QUALIFICATIONS What we need
- 4+ years experience in ETL development, specifically designing ETL/ELT automated data transformation processes.
- Expertise in SQL (on multiple database platforms).
- Expertise developing with Python, specifically working with packages like Pandas, dask, SQLAlchemy, boto3, and NumPy.
- Proficiency in designing and implementing database structures, ideally with Postgres.
- Proficiency acquiring, organizing, cleansing and manipulating large amounts of data.
- Proficiency in developing automated data validation test scenarios and scripts.
- What we’d really love Experience with the following are desired:
- Apache Airflow and AWS services like RDS, S3, and ECR/ECS.
- Apache Spark experience is a plus. Experience with RESTful APIs.
- Experience supporting self-service reporting tools such as Tableau, Looker, Periscope, Power BI, Spotfire, etc.
- Familiarity with financial systems integrations (SAP, Concur, Oracle, etc.) a plus.
- Experience working with financial transactions, PII, or in a regulated industry is a plus. An entrepreneurial spirit and experience with young companies is ideal.
