Data Engineering
Norstella’s brands are helping the world’s top pharmaceutical, medtech and clinical research organizations connect the dots from pipeline to patient in ways no one else can. It’s changing the industry – and it’s changing lives.
THE TEAM
Building on this success, we know there is more scope for data acquisition, data pipeline orchestration. advanced data processing, real time data ingestion and data science. The Data Engineering team consists of a blend of data engineers and data scientists with wide-ranging skills located across the globe.
As part of our growth, we are looking for new data engineers to help building data engineering pipelines and to support the production of new products. We have multi-functional teams consisting of pharmaceutical industry experts, R&D, data engineering and data scientists to rapidly prototype new visualisations and interactive reports using both our existing and newly acquired datasets.
SCOPE OF THE ROLE
In this role as a Data Engineer, you will:
• Design and optimization of database solutions that sit upstream of main production environment.
• Creation of optimized upstream database systems in Apache Pinot, RDS Databases and Redshift databases.
• Pulling, processing and structuring new data sets, from public and private sources, e.g. claims data.
• Automate data pipelines using Step Function, Airflow and AWS serverless compute services.
• Develop & maintain data publication and synchronization processes, supporting Norstella Data platform (NDP).
• Productionizing Data Warehouse for all data platforms.
HOW YOU'LL SUCCEED
You will work with RWD team to understand a problem which is hindering developing and releasing effective new data products which we believe we can help with. After understanding the problem and how it can be solved, you will work with the team to provide carefully curated and transformed data for the product.
The data sources may be already available in database, and simply require dedicated views, or be more complex such as ingestion of external datasets, use of Data acquisition Framework (DAF) to gather data, or to join with data from other companies within the Norstella group. Working with the broader product team, this dataset will be continuously updated, cleaned, and refined.
Data Engineering
After iterating the design with the visualisation team, the prototype will be demoed and if suitable converted into a full production product. This will involve productionizing the pipeline using Step Function / Airflow and other AWS cloud services, ensuring it is automatically refreshed with data if required, and putting in place monitoring and alerts so any issues can be identified and solved rapidly.
WHAT IT TAKES
• Bachelor’s degree in computer science, similar technical field of study or equivalent practical experience.
• Excellent SQL querying skills and stored procedure creation skills.
• Experience handling large data sets in formats such as XML, JSON and CSV.
• Building data pipelines.
• Good Python knowledge, with a focus on data manipulation.
• AWS Data services (S3, Glue, MWAA, Lambda, Step Functions, Secrets Manager, SQS, SES, etc.)
• Superior communication skills, friendly approach to business and the ability to grow and adapt as business does.
• Finger on the pulse with new & emerging technologies.
Desirable:
• AWS certifications.
• Redshift / Any Columnar / Any RDS certifications.
• Experience with data visualisation tools such as Tableau, Power BI or QuickSight.