Big Data Engineer – come work with us- a successful high-tech company??
We are looking for a Big Data Engineer to join our growing team of data experts.
The hire will be responsible for designing, implementing, and optimizing ETL processes and data pipeline flows within the our system.
The ideal candidate has experience in building data pipelines and data transformations enjoy optimizing data systems and building them from the ground up. The Big Data Engineer will support our data scientists with the implementation of the relevant data flows based on the data scientist’s features design.
They must be self-directed and comfortable supporting multiple production implementations for various use cases, part of which will be conducted on-premise at customer locations.
? Implement and maintain data pipeline flows in production within our system based on the data scientist’s design.
? Design and implement solution-based data flows for specific use cases, enabling applicability of implementations within our product.
? Building a Machine Learning data pipeline.
? Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
? Work with product, R&D, data, and analytics experts to strive for greater functionality in our systems.
? Train customer data scientists and engineers to maintain and amend data pipelines within the product.
? BSc degree or higher in Computer Science, Statistics, Informatics, Information Systems, Engineering, or another quantitative field.
? 1+ years of hands-on experience in working with Apache Spark cluster
? 1+ years of Hands-on experience and knowledge of Spark scripting languages: PySpark/Scala/Java/R
? 2+ years of Hands-on experience with SQL
? 1+ years of experience with data transformation, validations, cleansing, and ML feature engineering in a Big Data Engineer role
? Experience working with and optimizing ‘big data’ data pipelines, architectures, and data sets.
? Willingness to travel abroad to customers as needed (up to 25%)