DATA ENGINEER
The Data Engineer supports data collection, transformation, and loading efforts across the center, including data deriving from and transferred to statistical as well as geographic information system workflows. This involves establishing ETL workflows to support projects across the center; identifying and evaluating data sources for quality, reliability, and appropriateness; and migrating and integrating datasets as needed for analysis. Assist in the design of data collection and analysis efforts with a focus on preventing redundancies, improving quality, and developing systems to support longitudinal data collection and analysis.
Fundamental Responsibilities
- Performs data management tasks, including experienced data modeling, conversion, de-duplication, migration, and identification and repair of data quality issues.
- Designs, develops, and implements custom data systems and reconciliation tools, processes, rules, solutions etc. to validate data, match/merge, and upload batch lists.
- Creates and tunes complex stored procedures and queries for data management and extraction.
- Designs and builds out technical software mechanisms to accommodate multiple integrations accurately based on complex rules and custom solutions.
- Ensures documentation and security standards/protocols are recognized and followed.
- Provides experienced troubleshooting and problem analysis/resolution for data related issues; performs experienced scripting and modifications of application and products for corrective action.
- Researches and stays up-to-date with data engineering best practices and approaches; stays abreast of latest security threats and risks to proactively address potential exposures.
Department Specific Responsibilities
- Development, testing, and integration of ETL routines (including those for geospatial data) using ETL tools and external programming/scripting languages as necessary.
- Automating GIS analysis tasks using Python, ModelBuilder, FME, T-SQL and/or SSIS.
- Creating and maintaining the organization data dictionary as well as technical documentation for source-to-target mapping.
- Assisting in production support by resolving source data issues and refining transformation rules to align with center objectives.
- Ensure accuracy & integrity of data through analysis of application coding deliverables, insuring adequate documentation & problem resolution.
- Review and approval of analysis & translation of functional specifications and change requests into technical specifications
- Oversight of project document management of ETL technical system specifications, processes flows, unit tests and results.
- Ensures staff’s support of all ETL jobs for schedules and maintain compliance in development for effective project life cycle.
- Manage internal resources to ensure that individuals assigned to projects have the specific skills necessary to run established workflows.
Qualifications
Education
Required
- Bachelor’s degree
Preferred
- Degree in computer science, information science, or a related field
Work Experience
Required
- 2 years of data management, engineering, or related experience
Combinations of related education and experience may be considered.
Skills
Required
- Proficient communication skills
- Maintains a high degree of professionalism
- Demonstrated time management and priority setting skills
- Demonstrates a high commitment to quality
- Possesses flexibility to work in a fast paced, dynamic environment
- Seeks to acquire knowledge in area of specialty
- Highly thorough and dependable
- Demonstrates a high level of accuracy, even under pressure
Preferred
- Strong quantitative and geospatial data manipulation skills
- Advanced SQL coding skills for data transformations, profiling, and query tasks.
- Advanced working knowledge of Esri ArcGIS software
- Advanced working knowledge of at least 2 of the following: Python, ModelBuilder, FME
- Experience with SQL Server and SSIS (SQL Server Integration Services)
- Data deduplication strategies and implementation.
- Experience with data warehousing
- In-depth knowledge of relational databases such as SQL Server, Oracle, Access, and PostgreSQL
- Able to work independently and collaboratively as part of a project team and demonstrates initiative.
- Five years of experience with Extract, Transform, and Load (ETL) development or data modeling
- Experience with database management and working with large quantitative and geospatial datasets from disparate local, state, and federal sources, including census, American Community Survey, and other community-level data sets;
- Ability to manage and prioritize multiple assignments in a fast-paced environment, work independently, manage relationships with Polis staff members, show good judgment, and seek out support or guidance where appropriate.