Data Engineer

Role Responsibilities

If you are a data engineer who enjoys working with a small business team, and you want to make a difference, this job might be for you!

Reporting to the Manager – Data Operations, the Data Engineer 

1. Support design and maintenance of Data pipeline , data analytics workflows and data product workflows

• Analyze and organize raw data;
• Combine raw information from variety of different sources within and across data domains;
• Build data systems and pipelines which support defined business data products;
• Prepare data for prescriptive and predictive modeling;
• Build algorithms and prototypes, which deliver desired business value, for effective and efficient operations;
• Develop analytical tools and programs when required to support business customers; and
• Support synthetic data creation and data anonymization.

2. Contribute to Data Architecture
 

• Evaluate business needs and objectives;
• Interpret technology trends and patterns;
• Explore and incorporate ways to enhance data quality and reliability into defined data services;
• Design analytical tools and programs to support defined data services; and
• Design integration and orchestration patterns to operate data products effectively and efficiently with available resources.

3. Contribute and support Data Operations

• Monitor and report on data pipeline operations;
• Conduct complex data analysis and report on results;
• Identify opportunities for additional valuable data acquisition; and
• Collaborate with data scientists, data analysts, and data architects on several projects simultaneously.

4. Support Data Operations Manager and other leaders to accomplishing the mandate of the DataOps team
 

• Support the cultural shift towards data as a strategic asset;
• Foster data engineering innovation as well as continuous improvement;
• Present options, recommendations, and solutions to management regarding data engineering and design;
• Ensure effective stakeholder engagement within the areas of responsibility ensuring an effective solution balance of enterprise and client centric needs; and
• Initiate and maintain effective working relationships and partnerships with various internal and external stakeholders across the organization.
• Develop and optimize ETL pipelines, managing data warehouse and orchestration platforms, and cloud storage to reliably move data from multiple source systems, assessing existing infrastructure, and developing a roadmap for infrastructure improvement.

Role Responsibilities (Cont'd)

These are your goals:

Data Operations team is developed to meet AE's operational and analytic needs

• Identify and document requirements for new data infrastructure, working with data  infrastructure leads, data architects, data modellers, and business users to ensure outcomes are clear and shared.
• Build data pipelines from one or more sources to cloud storage and data warehouse in a customized way, including orchestration, data masking, testing, reviewing, troubleshooting, scheduling, and documenting.
• Organize structured and unstructured data into analyst-friendly tables in the centralized data cloud.
• Develop messaging and alerting systems so that failure and error reporting happen in an automated, trusted way.
• Maintain the tooling libraries that enable automation and quality of the data warehouse: custom operators and scripts for orchestration, continuous integration and deployment, code deployment process, and linting; maintaining source to target mapping documents, metadata and data dictionary.
• Enable analysts and modellers to work effectively by supporting the tools they use, including Data Build Tools (DBT) for data transformations inside the centralized data cloud.

Data engineering and data modelling functions are backed up

• Cross train with data modelling and reporting colleagues to ensure business continuity.
• Draft, test, and review data model scripts.
• Help develop and maintain data model documentation including standards.
• Draft and review reports, visualizations, and dashboards.
• Help develop and maintain reporting and visualization documentation including standards.

 
AE improves data operations capacity and data management maturity

• Support unit staff to develop data infrastructure building skills including teaching the basics, guiding junior staff through complex cases, and validating that pipelines work as intended.
• Support unit and department colleagues to read and understand data management documentation.
• Support the Manager and Director to develop and hire to additional data engineering positions as needed.
• Help to develop and socialize with colleagues and stakeholders an evolving data management framework and ongoing data operations functions, roles, and systems.
• Support the Manager and Director to prepare briefings and presentations about data management and governance.
• Best practices guide infrastructure development and planning.
• Work with the Manager, Data Operations to establish AE's data infrastructure building standards, developing documentation and processes to support junior and future data engineering staff in the unit.
• Identify and prioritize building and maintenance work, taking in to account department business priorities, technical dependencies, and unit coordination and capacity.
• Develop and maintain a roadmap for development and improvement of infrastructure and standards.
• Build a library of data infrastructure documentation and resources for unit and Department staff.

Please click on this link to view the job description for this position.

APS Competencies

Competencies are behaviors that are essential to reach our goals in serving Albertans. We encourage you to have an in depth understanding of the competencies that are required for this opportunity and to be prepared to demonstrate them during the recruitment process.

This link will assist you with understanding competencies:
https://www.alberta.ca/system/files/custom_downloaded_images/psc-alberta-public-service-competency-model.pdf.

In this role, you will contribute to our team by demonstrating the following behavioral competencies:

• Systems Thinking: Understands and acts with an awareness of the ministry’s culture, business, and environment; the government’s direction and political climate; and stakeholder issues across Alberta’s economic, innovation, and learning systems. Demonstrates strong analytical skills and sound judgment to manage high-profile functions involving sensitive information.
  
  
• Develop Networks: Developing collaborative environments by focusing on authentic relationships that leverage expertise, ingenuity, and effort from staff and colleagues across the Ministry, Government, and post-secondary system.
  
  
• Creative Problem Solving: Identifies and brings together the right people to solve complex problems and find solutions; seeks out alternative ways to improve complex systems or processes to better achieve desired outcomes.
  
  
• Drive for Results: Plans for and delivers results, from concrete deliverables to process improvements, managing resources effectively, and ensuring attention to detail.
  
  
• Build Collaborative Environments: Strong track record in establishing and maintaining effective working relationships with internal and external stakeholders.

Qualifications

Applicants must have a University graduation (in computer science or a related discipline plus two years related experience; or equivalent as described.

Equivalency: A related two-year diploma in computer technology or a related discipline from a recognized post-secondary institution and four years related experience; or a related one-year certificate from a recognized post-secondary institution and five years related experience

Job-specific experience, technical competencies, certification and/or training:

Knowledge:

• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases;
• Experience building and optimizing data pipelines, architectures and data sets;
• Experience building processes that support data transformation, data structures, metadata, master data, dependency and workload management;
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement;
• A successful history of manipulating, processing and extracting value from large disconnected datasets;
• Working knowledge of message queuing, stream processing, and highly scalable data stores; and
• Experience supporting and working with cross-functional teams in a dynamic environment.

Skills:

• Strong analytic skills related to working with structured and unstructured datasets;
• Strong organizational skills to manage multiple complex tasks effectively;
• Strong negotiation and facilitation skills;
• Sound problem solving skills - identification, analysis, and evaluation of problems, opportunities, and associated outcomes;
• Strong communication skills: written, verbal, non-verbal and presentation;
• Well-developed project and time management skills;
• Able to work in both a team environment and independently; and
• Strong relationship building, client services, and partnership skills.

 
Technical Competencies, certification and/or training:

• Big data tools such as Snowflake, Hadoop, Spark, Kafka, etc.;
• Relational SQL and NoSQL databases, including Postgres and Cassandra;
• Data pipeline and workflow management tools such as Data Factory, DBT,SSIS etc.;
• Data quality tools such as Purview, Collibra, Informatica etc.;
• Cloud services such as Azure, AWS;
• Stream-processing systems such as Storm, Spark-Streaming, etc.;
• Object-oriented/object function scripting languages such as R, Python, Java, C++, Scala, etc.;
• Data visualization tools such as Power BI, Tableau, SAS, etc. is an asset; and

The following practical experience is desirable:

• Agile delivery approaches such as DataOps, CI/CD, or DevOps.
• Data modelling.
• Visualization tools such as PowerBI, SSRS, Tableau.

Minimum recruitment standards outline:
https://www.alberta.ca/alberta-public-service-minimum-recruitment-standards.

Notes

This job is performed in hybrid mode, with 3 days per week in-person at the downtown Edmonton office in Alberta. Applicants from outside the Edmonton area should consider if they are willing to relocate.

Applications with a cover letter will be given preference. Applicants should summarize information that clearly and concisely demonstrates how their qualifications meet the advertised requirements, including education, experience and relevant examples of required competencies

Costs to obtain required documents or checks or to travel for an interview are the candidate’s responsibility. Out-of-province applicants can obtain the required documents/checks from the province they currently reside in.

Links and information on what the GOA has to offer to prospective employees:

• Working for the Alberta Public Service - https://www.alberta.ca/advantages-working-for-alberta-public-service.aspx
• Public Service Pension Plan (PSPP) - https://www.pspp.ca
• Alberta Public Service Benefit Information - https://www.alberta.ca/alberta-public-service-benefits
• Professional learning and development - https://www.alberta.ca/professional-development-support-directive
• Research Alberta Public Service Careers tool – https://researchapscareers.alberta.ca
• Positive workplace culture and work-life balance
• Opportunity to participate in flexible work arrangements such as working from home up to two days per week and modified work schedule agreement
• Leadership and mentorship programs