Title:Data Engineer
Job Duties:
- Design, develop, test, and implement business rule. Gather requirements and analyze business processes.
- Create pipelines, data flows, complex data transformations and manipulations using ADF and PySpark with Databricks.
- Construct models in cloud environment which includes automated processes, CI/CD pipelines, monitoring/alerting, and troubleshooting issues.
- Establish audit, balance and control framework using SQL DB audit tables to control ingestion, transformation, and load process in Azure.
- Develop Databricks Python notebooks to Join, filter, pre-aggregate, and process files stored in Azure data lake storage Gen1/Gen2.
- Utilize Azure Logic Apps to develop workflows which can send/alert notifications, and Azure DevOps to build and release various codes in different environments.
- Build data pipelines using Azure Service to load data from Legacy SQL server to Azure Data Base using Data Factories, API Gateway Services, SSIS Packages, Talend Jobs, custom .Net and Python codes.
- Design and maintain power BI Data models using multiple DAX expressions to build calculated columns and measures.
- Build business intelligence dashboards, analytical reports and data visualizations using power BI.
- Must have knowledge with regards to Azure authentication mechanisms such as Service principal, Managed Identity, Key vaults.
- Required tools and utilities: Teradata, Oracle, SQL, JSON, and XML.
Other:Full Time. Travel involved to unanticipated client locations within US.
Requirements:Master's degree in Computer Science or Equivalent. Any suitable combination of education, training or experience accepted.
Location:St. Peters, MO.
Respond to:HR - Code BKK, Aspyretech, Inc., 4137 Mexico Road, St. Peters, MO, 63376.
This position is eligible for our employee referral program