Senior Data Engineer responsible for designing and validating data solutions, consolidating disparate data sources, documenting functional specifications, reviewing data models, supporting production deployments, investigating data quality issues, and collaborating with stakeholders and technology leads to ensure data completeness and lineage.
Responsibilities
- Big Data - Spark, Hive, Java, CDP, 6 Years + with 3 Years of Dev experience on Big Data
- Analyze data requirements and identify disparate data sources required for consolidation and distribution.
- Document functional specifications and coordinate delivery of the same with technology team.
- Review logical and conceptual data models in alignment with business requirements.
- Work with the stakeholders to understand and gather requirements and produce business specifications
- Validate solution implementations and ensure they meet business and functional requirements.
- Provide production deployment support and investigate data quality issues.
- Work with various technology leads to ensure the gaps in the data completeness or accuracy are bridged.
Qualifications
- Subject matter expertise in financial industry - wholesale loans /lending business OR Capital Markets or Finance or Risk Reporting
- Strong hands on experience with database and SQL is required.
- Excellent documentation, analytical skills to produce process flow diagrams, business modelling, and functional design.
- Proficiency in MS Office (Word, Excel, Visio, PowerPoint) with extensive experience using Excel for data analysis.
- Experience with Data tracing/ Lineage efforts
- Knowledge of logical and physical data model
Photon Vancouver, British Columbia, CAN Office
900-885 W Georgia Street , , Vancouver, BC , Canada, V6C 3H1
Similar Jobs
Agency • Information Technology
Senior Java Big Data Developer with 8+ years building applications on Cloudera/Hadoop ecosystems using Spark and MapReduce. Responsible for designing performant Java (1.8+) solutions, data analysis of large datasets, unit testing (JUnit), and collaborating with/managing global technology teams. Preferred skills include Unix/Python scripting and familiarity with Hive, Impala, YARN, and Kafka.
Top Skills:
SparkClouderaHadoopHiveImpalaJava 1.8JunitKafkaMapreduceNoSQLPythonUnix ShellYarn
Agency • Information Technology
Design, develop, test, and deploy high-performance Apache Spark/Scala data processing applications and ETL pipelines on Cloudera (CDH). Optimize Spark and platform performance, build data pipelines using HDFS, Hive, Impala, HBase, and Kafka, ensure data integrity and security, collaborate with data scientists and analysts, and implement version control and CI/CD for deployments.
Top Skills:
SparkCi/CdCloudera (Cdh)FlumeGitGitlabHbaseHdfsHiveImpalaJenkinsKafkaNifiOoziePostgresScalaSpark SqlSqoop
Agency • Information Technology
Build and maintain responsive Angular UI components, collaborate with UX and backend teams, implement REST APIs and Python backend scripts, integrate basic ML models into data workflows, and participate in the full SDLC within Agile teams while ensuring code quality and performance.
Top Skills:
AngularCi/CdCSS3DjangoFastapiFlaskGitHTML5JavaScriptJSONMachine LearningPythonRestTypescript
What you need to know about the Vancouver Tech Scene
Raincouver, Vancity, The Big Smoke — Vancouver is known by many names, and in recent years, it has gained a reputation as a growing hub for both tech and sustainability. Renowned for its natural beauty, the city has become a magnet for professionals eager to create environmental solutions, and with an emphasis on clean technology, renewable energy and environmental innovation, it's attracted companies across various industries, all working toward a shared goal: advancing clean technology.
