Senior Software Engineer (Big Data)

Job description / Role

Our client an international technology provider is currently looking for a Senior Software Engineer for their Big Data project. In this role, you will be responsible for participating as member of a cross functional team working autonomously on technology development and problem resolution. The role involves the design, development, implementation and maintenance of technology solutions and products.

Responsibilities:

• Lead technical design and build for small to medium sized solutions in a team. Translate functional and non-functional requirements into fit for purpose technical design.
• Debug issues of complexity and follows design documents with minimal or no supervision.
• Code review by peers/technical leads demonstrates good coding principles.
• Software engineering tasks or problems of moderate to high scope.
• Translate business solution requirements into potential technical solutions based on technical design.
• Ensure solution performance, business edge cases and security related issues while developing software.
• Work on problems of diverse scope where analysis of data requires evaluation of identifiable factors.
• Demonstrate good judgment in selecting methods and techniques for obtaining solutions.
• For Data and Analytics solutions, conduct data analysis activities such as source system analysis, data dictionary collection, data profiling and source-to-target mapping to ensure solutions deliver on business needs.
• Automate unit tests and carries out testing independently.
• Work with senior technical engineers in the technical design process by contributing in the analysis of technical application requirements.
• Update data inventories and registries as required to keep metadata and data lineage up-to-date, following agreed Data Governance standards, guidelines and principles.
• Troubleshoot issues, fixes defects that are moderate to high complexity and carry out testing independently.
• Shadows senior engineers on design and architecture components and collaborate with members of the cross functional team to identify areas of inefficiency and propose solutions.

Requirements

Requirements:
• Minimum 5+ years of Developing and testing experience in the Apache Hadoop framework, Azure cloud, Lambda architecture and Data Lake.
• Experience in Big Data and batch/real-time ingestion and analytical solutions leveraging transformational technologies.
• Experience in Java, spring, Elasticsearch and Cassandra.
• Understand the source systems data, gather & able to transfer the data into HDFS as RAW and Decomposed layer (including writing Oozie workflow, coordinators, Sqoop and Flume).
• Ability to identify Unit testing.
• Experience and knowledge of a wide variety of testing methods and tools covering functional, performance, regression, system integration, smoke testing of Business Intelligence, ETL, BigData ( Ingestion, Modeling, Serving layer) and Business Analytical landscape.
• Understand the source systems data, gather & able to transfer the data into HDFS as RAW and Decomposed layer (including writing Oozie workflow, coordinators, Sqoop and Flume).
• Exposure in handling to massive quantities of data by taking advantage of both batch and speed methods.
• Strong knowledge of and experience with statistics; potentially other advanced math as well and good working knowledge of SQL ability.
• Deep knowledge in data mining, data modelling, data processing, machine learning, natural language processing or information retrieval.
• Experience in processing large amounts of structured / unstructured data.
• Capability to design and document solutions independently.

Disclaimer: Black Pearl will never ask for money or any form to charge our candidates just to process or consider their application for any of our available vacancies. If you happen to receive such request from any members of our staff or other individuals claiming to be part of Black Pearl, please do call our office.

About the Company

Black Pearl is a human resources and recruitment agency that provides a holistic business and talent solution in a progressive, dynamic, and efficient way.

We pride ourselves as a unique black pearl and strive to delight our clients with a personalized partnership that helps them propel towards success.

Whether you need to find top talent, your next great job opportunity, or a consulting solution for managing your business, engaging employees, evolving your work culture, planning your people strategy, or resourcing challenges, we can help.