Big Data Engineer

{{ flashMessage.message }}

JOB DESCRIPTION / ROLE

Employment: Full Time

We are looking for a Big Data Engineer that will work on the collecting, storing, processing, and analyzing of huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company.

• Working with and extracting value from large, disconnected and/or unstructured datasets.
• Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities.
• Demonstrated ability to build processes that support data transformation, data structures, metadata, dependency and workload management.
• Project manage and work with cross-functional teams.
• Building and optimizing ‘big data’ data pipelines, architectures and data sets.
• Performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Integrate data from multiple data sources
• Experience with the following tools and technologies:
• Stream-processing systems such as Storm and Spark-Streaming
• API integration with bigdata
• Object-oriented/object function scripting languages such as Python, Java, C++, etc.

REQUIREMENTS

• Graduate degree in Computer Science, Information Systems or equivalent quantitative field and 6+ years of experience.
• Experience working with and extracting value from large, disconnected and/or unstructured datasets.
• Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities.
• Demonstrated ability to build processes that support data transformation, data structures, metadata, dependency and workload management.
• Strong interpersonal skills and ability to project manage and work with cross-functional teams.
• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
• Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Experience with integration of data from multiple data sources
• Experience with the following tools and technologies:
• Cloudera, Hadoop, Spark core, Spark SQL, Scala, Hive, Sqoop, Kafka.
• Data ingestion [Apache NiFi]
• Big Data querying tools, Pig, Hive, HBase, Impala, YARN and HDFS
• Zookeeper, Ranger, Atlas, Kudu
• Relational SQL and NoSQL databases
• Data pipeline/workflow management tools such as Azkaban and Airflow
• Stream-processing systems such as Storm and Spark-Streaming
• API integration with bigdata
• Object-oriented/object function scripting languages such as Python, Java, C++, etc.
• Experience with ETL Processes

ABOUT THE COMPANY

Global Knowledge is the worldwide leader in IT and business training. We deliver via training centers, private facilities, and the Internet, enabling our customers to choose when, where, and how they want to receive training programs and learning services.

Our core training is focused on Cisco, Microsoft, Nortel, and Project Management. Our IT courses include networking, programming, operating systems, security, and telephony. Our business courses feature project management, professional skills, and business process curriculum, including ITIL. Our more than 700 courses span foundational and specialized training and certifications.

In 2006, we acquired Azlan Training, which established us as Europe’s largest IT training provider. In 2007, we expanded our global footprint to include IT training, consulting, and services in the Middle East and Africa through the acquisition of Synergy Professional Services.

Advertise Here
INSTALL APP
×